Portal:Cloud VPS/Admin/Runbooks/GaleraNoRecentWrites

From Wikitech

The GaleraNoRecentWrites alert fires when the Galera wsrep_last_committed status variable has not increased in the last hour.

The procedures in this runbook require admin permissions to complete.

Error / Incident

Galera has applied no writes in the past hour.

Debugging

Some tips:

  • Look at the value of the variable on all nodes

Common issues

Single node behind others, all nodes stuck

Once a single Galera node had a lower value than all others. In that case we depooled the node which moved traffic to other nodes, those two nodes then started having new writes applied.

Related information

Support contacts

Old incidents