Kafka Java Clients Monitoring¶
Kafka Java Producers and Consumers Monitoring Dashboard
Tags¶
PrometheusKafka
Panels¶
Overview¶
| Name | Description | Thresholds | Repeat |
|---|---|---|---|
| Producers Overview | Number of records and bytes produced per second for a specific topic or across all topics. | ||
| Consumers Overview | Number of records and bytes consumed per second for a specific topic or across all topics. |
Resources Metrics¶
| Name | Description | Thresholds | Repeat |
|---|---|---|---|
| CPU Usage | CPU usage by pod. | Default: Mode: absolute Level 1: 80 |
|
| Memory Usage | Memory usage by pod. | Default: Mode: absolute Level 1: 80 |
|
| Heap Memory Usage | Heap memory usage by pod. | Default: Mode: absolute Level 1: 80 |
|
| Network Rate | Network Rate: transmitted (tx) and received (rx). | Default: Mode: absolute Level 1: 80 |
Producers Metrics¶
| Name | Description | Thresholds | Repeat |
|---|---|---|---|
| Producer Records Rate | The rate of records that successfully processed by Producer and written to Kafka topics. | Default: Mode: absolute Level 1: 80 |
|
| Producer Records Bytes Rate | The rate of bytes that successfully processed by Producer and written to Kafka topics. | Default: Mode: absolute Level 1: 80 |
|
| Producer Request Rate | Average number of requests sent per second. | Default: Mode: absolute Level 1: 80 |
|
| Producer Errors Count | The number of errors during producing. | Default: Mode: absolute Level 1: 80 |
|
| Average Producer Record Queue Time | The average time in milliseconds record batches spent in the send buffer. | Default: Mode: absolute Level 1: 80 |
|
| Average Producer Request Latency | Average request latency (in ms). | Default: Mode: absolute Level 1: 80 |
|
| Producer Errors Rate | The rate of errors during producing. | Default: Mode: absolute Level 1: 80 |
|
| Average Records Size | The average record size. | Default: Mode: absolute Level 1: 80 |
|
| Producer Response Count | Number of responses received. | Default: Mode: absolute Level 1: 80 |
|
| Topics Errors Count | The number of errors during producing by topics. | Default: Mode: absolute Level 1: 80 |
|
| Average Records Batch Size | The average record batch size. | Default: Mode: absolute Level 1: 80 |
|
| Producer Response Rate | Average number of responses received per second. | Default: Mode: absolute Level 1: 80 |
|
| Waiting Threads | The number of threads in waiting state. | Default: Mode: absolute Level 1: 80 |
|
| Average IO Wait Producer Time | Average length of time the I/O thread spent waiting for a socket (in ns). | Default: Mode: absolute Level 1: 80 |
|
| Average Compression Rate | Average compression rate of batches sent. | Default: Mode: absolute Level 1: 80 |
|
| In-Flight Requests | The number of in-flight requests. | Default: Mode: absolute Level 1: 80 |
|
| Producer Outgoing Bytes Rate | The average number of outgoing/incoming bytes per second. | Default: Mode: absolute Level 1: 80 |
Consumer Metrics¶
| Name | Description | Thresholds | Repeat |
|---|---|---|---|
| Consumer Records Rate | Average number of records consumed per second for a specific topic or across all topics. | Default: Mode: absolute Level 1: 80 |
|
| Consumer Records Bytes Rate | Average number of bytes consumed per second for a specific topic or across all topics. | Default: Mode: absolute Level 1: 80 |
|
| Consumer Request Rate | The rate of requests sent. | Default: Mode: absolute Level 1: 80 |
|
| Consumer Lag | Number of messages consumer is behind producer on this partition. | Default: Mode: absolute Level 1: 80 |
|
| Fetch Rate | The number of fetch requests per second from the consumer. | Default: Mode: absolute Level 1: 80 |
|
| Average Consumer Request Size | The average Consumer request size. | Default: Mode: absolute Level 1: 80 |
|
| Consumer Rebalance Count | Number of rebalances and failed rebalances. | Default: Mode: absolute Level 1: 80 |
|
| IO Wait Consumer Time Rate | Average length of time the I/O thread spent waiting for a socket (in ns). | Default: Mode: absolute Level 1: 80 |
|
| Average Consumer Records Per Request | The average number of records per request. | Default: Mode: absolute Level 1: 80 |
|
| Consumer Rebalance Rate | The rate of rebalances and failed rebalances. | Default: Mode: absolute Level 1: 80 |
|
| Consumer Incoming Bytes Rate | The number of bytes that successfully processed by Consumer and written to Kafka topics per second. | Default: Mode: absolute Level 1: 80 |
|
| Average Time Between Poll | The average time between poll in ms. | Default: Mode: absolute Level 1: 80 |
Listener Metrics (Spring)¶
| Name | Description | Thresholds | Repeat |
|---|---|---|---|
| Listener Calls Rate | Shows per-second rate of listener calls count | Default: Mode: absolute Level 1: 80 |
|
| Listener Failures Rate | Shows the count of Kafka listener failures | ||
| Average Listener Processing Duration | Shows the average time of successful call processing by listener | Default: Mode: absolute Level 1: 80 |