Data Platform/Systems/EventLogging/Monitoring
Appearance
Grafana
Here's some really cool Grafana dashboards that display our Graphite metrics:
- EventLogging Dashboard, shows hitrates of all events by schema, status of Kafka brokers, error counts and more.
- EventLogging-schema Dashboard, focus on an individual schema. Useful for embedding and inter-dashboard links from graphs showing the actual EventLogging data. Example: performance-metrics.
Graphite
Raw metrics can be browsed at https://graphite.wikimedia.org/ (under Metrics -> eventlogging).
We publish 4 types of counts to graphite:
- Overall counts
- Per schema counts
- Server side counts
- Client side counts
Within the overall counts, there are 4 submetrics:
- Raw counts: Number of all events that reach the system.
- Valid counts: Number of events that pass validation.
- insertAttempted: Number of events that get queued up for insertion to MySQL database.
- inserted: Number of events that get actually inserted into MySQL database.