Session Agenda
Building Great Dashboards
Two Types Of Dashboards
Two Sets of Widgets
Timeboards
Screenboards
Two Different Goals
ElasticSearch Metrics
ElasticSearch Metrics
| Metric | Description |
|---|---|
| elasticsearch.active_primary_shards (gauge) |
The number of active primary shards in the cluster. shown as shard |
| elasticsearch.active_shards (gauge) |
The number of active shards in the cluster. shown as shard |
| elasticsearch.breakers.fielddata.estimated_size_in_bytes (gauge) |
The estimated size in bytes of the field data circuit breaker. shown as byte |
| elasticsearch.breakers.fielddata.overhead (gauge) |
The constant multiplier for byte estimations of the field data circuit breaker. |
| elasticsearch.breakers.fielddata.tripped (gauge) |
The number of times the field data circuit breaker has tripped. |
| elasticsearch.breakers.parent.estimated_size_in_bytes (gauge) |
The estimated size in bytes of the parent circuit breaker. shown as byte |
| elasticsearch.breakers.parent.overhead (gauge) |
The constant multiplier for byte estimations of the parent circuit breaker. |
| elasticsearch.breakers.parent.tripped (gauge) |
The number of times the parent circuit breaker has tripped. |
| elasticsearch.breakers.request.estimated_size_in_bytes (gauge) |
The estimated size in bytes of the request circuit breaker. shown as byte |
| elasticsearch.breakers.request.overhead (gauge) |
The constant multiplier for byte estimations of the request circuit breaker. |
| elasticsearch.breakers.request.tripped (gauge) |
The number of times the request circuit breaker has tripped. |
| Metric | Description |
|---|---|
| elasticsearch.cache.field.evictions (gauge) |
The total number of evictions from the field data cache. shown as eviction |
| elasticsearch.cache.field.size (gauge) |
The size of the field cache. shown as byte |
| elasticsearch.cache.filter.count (gauge) |
The number of items in the filter cache. shown as item |
| elasticsearch.cache.filter.evictions (gauge) |
The total number of evictions from the filter cache. shown as eviction |
| elasticsearch.cache.filter.size (gauge) |
The size of the filter cache. shown as byte |
| elasticsearch.cluster_status (gauge) |
The elasticsearch cluster health as a number: red = 0, yellow = 1, green = 2 |
| elasticsearch.docs.count (gauge) |
The total number of documents in the cluster across all shards. shown as document |
| elasticsearch.docs.deleted (gauge) |
The total number of documents deleted from the cluster across all shards. shown as document |
| elasticsearch.fielddata.evictions (gauge) |
The total number of evictions from the fielddata cache. shown as eviction |
| elasticsearch.fielddata.size (gauge) |
The size of the fielddata cache. shown as byte |
| elasticsearch.flush.total (gauge) |
The total number of index flushes to disk since start. shown as flush |
| elasticsearch.flush.total.time (gauge) |
The total time spent flushing the index to disk. shown as second |
| Metric | Description |
|---|---|
| elasticsearch.fs.total.available_in_bytes (gauge) |
The total number of bytes available to this Java virtual machine on this file store. shown as byte |
| elasticsearch.fs.total.disk_io_op (gauge) |
The total I/O operations on the file store. shown as operation |
| elasticsearch.fs.total.disk_io_size_in_bytes (gauge) |
Total bytes used for all I/O operations on the file store. shown as byte |
| elasticsearch.fs.total.disk_read_size_in_bytes (gauge) |
The total bytes read from the file store. shown as byte |
| elasticsearch.fs.total.disk_reads (gauge) |
The total number of reads from the file store. shown as read |
| elasticsearch.fs.total.disk_write_size_in_bytes (gauge) |
The total bytes written to the file store. shown as byte |
| elasticsearch.fs.total.disk_writes (gauge) |
The total number of writes to the file store. shown as write |
| elasticsearch.fs.total.free_in_bytes (gauge) |
The total number of unallocated bytes in the file store. shown as byte |
| elasticsearch.fs.total.total_in_bytes (gauge) |
The total size in bytes of the file store. shown as byte |
| elasticsearch.get.current (gauge) |
The number of get requests currently running. shown as request |
| elasticsearch.get.exists.time (gauge) |
The total time spent on get requests where the document existed. shown as second |
| elasticsearch.get.exists.total (gauge) |
The total number of get requests where the document existed. shown as request |
| elasticsearch.get.missing.time (gauge) |
The total time spent on get requests where the document was missing. shown as second |
| elasticsearch.get.missing.total (gauge) |
The total number of get requests where the document was missing. shown as request |
| elasticsearch.get.time (gauge) |
The total time spent on get requests. shown as second |
| elasticsearch.get.total (gauge) |
The total number of get requests. shown as request |
| Metric | Description |
|---|---|
| elasticsearch.http.current_open (gauge) |
The number of current open HTTP connections. shown as connection |
| elasticsearch.http.total_opened (gauge) |
The total number of opened HTTP connections. shown as connection |
| elasticsearch.id_cache.size (gauge) |
The size of the id cache shown as byte |
| elasticsearch.indexing.delete.current (gauge) |
The number of documents currently being deleted from an index. shown as document |
| elasticsearch.indexing.delete.time (gauge) |
The total time spent deleting documents from an index. shown as second |
| elasticsearch.indexing.delete.total (gauge) |
The total number of documents deleted from an index. shown as document |
| elasticsearch.indexing.index.current (gauge) |
The number of documents currently being indexed to an index. shown as document |
| elasticsearch.indexing.index.time (gauge) |
The total time spent indexing documents to an index. shown as second |
| elasticsearch.indexing.index.total (gauge) |
The total number of documents indexed to an index. shown as document |
See all of the metrics for Elasticsearch
What do you think the work metrics should be?
The server does something.
- Throughput Metrics: How many of those things did it do?
- Performance Metrics: How long does it take to do those things?
- Success Metrics: How many of those things were done successfully?
- Error Metrics: How many of those things were not done successfully?
What do you think the resource metrics should be?
The server does something and relies on a resource.
- Utilization Metrics: How busy was the resource while the server did its thing?
- Saturation Metrics: How long is the queue of things waiting for the server?
- Error Metrics: How many errors is the resource experiencing with regards to the server?
- Availability Metrics: How long is the resource available for?
Some Key Elasticsearch Metrics
| Category | Metric |
|---|---|
| Work:Throughput | elasticsearch.primaries.search.query.total |
| Work:Throughput | elasticsearch.primaries.search.query.current |
| Work:Throughput | elasticsearch.primaries.search.fetch.current |
| Work:Performance | elasticsearch.primaries.search.query.time |
| Resource:Utilization | jvm.mem.non_heap_committed |
| Resource:Utilization | jvm.mem.heap_committed |
| Resource:Availability | elasticsearch.initializing_shards |
| Resource:Saturation | elasticsearch.thread_pool.bulk.queue |
See https://www.datadoghq.com/blog/monitor-elasticsearch-performance-metrics/ for more
Building a Timeseries on a Dashboard
- Create a Timeboard or Screenboard
- Drag Timeseries to the canvas
- Select the metric
- Set the Scope (host, etc)
- Space Aggregation (avg, max, min, sum; by x)
- Time Aggregation (optional)(rollup)
- Functions
Building a Timeseries
Building a Timeseries
Building a Timeseries
Building a Timeseries
Building a Timeseries
Building Great Dashboards
