The dashboard must answer questions#
During an incident there is no time to interpret dozens of decorative charts. A useful dashboard shows what changed, who is affected and which resource is approaching its limit.
The four signals#
- Latency: how long work takes, including high percentiles.
- Traffic: how much work reaches the system.
- Errors: failed requests grouped by type and dependency.
- Saturation: remaining headroom in CPU, memory, connections and queues.
Deployment context#
Metrics should display versions and deployment events. A regression that begins exactly with a new version dramatically reduces the search space.
From service to dependency#
Start with the user symptom and move down one layer at a time: API, queue, database and external provider. Avoid building a different dashboard for every possible incident.
A simple criterion#
Every chart should support a decision: roll back, scale, limit traffic or investigate a dependency. If it never changes the operator’s action, it probably does not deserve space in the primary view.