Portal:Cloud VPS/Admin/Runbooks/GaleraDown

From Wikitech

The GaleraDown alert fires when mysqld exporter can't connect to the MariaDB instance on a cloudcontrol node.

The procedures in this runbook require admin permissions to complete.

Error / Incident

The datastore used by all OpenStack services is unreachable.

Debugging

Log in to the host in question, then:

  1. Check if you can log in to the MariaDB host manually:
    user@cloudcontrolXXXX ~ $ sudo -i mariadb -u root
    
  2. Check the mysqld-exporter logs:
    user@cloudcontrolXXXX ~ $ sudo systemctl status prometheus-mysqld-exporter.service
    

Common issues

Add here any new common issues you find.

Related information

Support contacts

Old incidents

Add here any new tasks for incidents you might encounter.