News/2023 ToolsDB upgrade

From Wikitech

In April 2023, ToolsDB was migrated to a new Cloud VPS instance, and is now running MariaDB 10.4 with Ceph-based storage.

Timeline

  • 2023-03 New MariaDB server created and set to replicate data from the old primary
  • 2023-04-06 New server promoted to primary, DNS records pointed to the new primary server (T333471)

What you need to do

Hopefully nothing. If your tool lost connectivity to the database, please try restarting it. Note: ideally all tools should automatically reconnect when they lose database connectivity.

What you should be aware of

The new version of MariaDB (10.4) introduced some breaking changes from the one we were running on the old server (10.1).

So far the ones listed below are the ones that we know have required manual changes from users:

Performance

The new storage is slightly less performant than the old one, because it's using network storage (Ceph) and no longer physically-attached storage. We believe the slight performance penalty is compensated by the vastly improved reliability and flexibility. We are not observing significant issues after the migration, but we expect some queries could be (marginally) slower.

Linkwatcher

We decided NOT to migrate the database s51230__linkwatcher (by far the biggest database in the old ToolsDB server) and instead move it to a dedicated Trove instance.