MariaDB/monitoring
< MariaDB
Icinga
Specific checks:
- Replication running (both IO and SQL thread)
- Replication lag
- Disk space
- Memory used
- Read_only status
- mariadb process
Metrics
Prometheus
Hosts to be monitored at prometheus are controlled by the instance and server inventory at Zarcillo (db1115). In order to update prometheus, hosts have to be inserted, updated or deleted from Zarcillo and then run /usr/local/sbin/mysqld_exporter_config.py on the prometheus hosts (e.g. prometheus1003 and prometheus1004 for eqiad). This scripts run automatically every 30 minutes to check for changes.
The standard mysqld-prometheus-exporter is used for most metrics.
Grafana
Relevant dashboards:
- MySQL aggregated: https://grafana.wikimedia.org/d/000000278/mysql-aggregated
- MySQL instance: https://grafana.wikimedia.org/d/000000273/mysql
- Replication lag: https://grafana.wikimedia.org/d/000000303/mysql-replication-lag
- MediaWiki LoadBalancer: https://grafana.wikimedia.org/d/G9kbQdRVz/mediawiki-loadbalancer
Orchestrator
Logstash/Kibana
Useful dashboards:
- DBQuery https://logstash.wikimedia.org/goto/3063b45795e5d484c75e0039f997ba5b
- DBError https://logstash.wikimedia.org/goto/dce505a8dac307f3b67923f9995d155c
- SlowQuery https://logstash.wikimedia.org/goto/1dda4970a91dd77ee4f322848aef3ae3
This page is a part of the SRE Data Persistence technical documentation
(go here for a list of all our pages)