Dumps/Snapshot hosts

From Wikitech
(Redirected from Snapshot3)

Snapshot (XML dumps generation) cluster information

Hardware

These hosts generate the sql/XML dumps or the "misc cron" dumps. For information about the hosts that serve them, see Dumps/Dump servers.

In eqiad:

  • snapshot1008: primary (other dumps), Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6130 16-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1010: primary and monitor, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1011: primary, Dell PowerEdge R440, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1012: primary, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1013: primary, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
  • snpashot1014: spare, Dell PowerEdge R440, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1015: spare, Dell PowerEdge R440, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1016: spare, Dell PowerEdge R650xs, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6326 16-core CPUs, 2 480GB SSDs in software raid 1
  • snapshot1017: spare, Dell PowerEdge R650xs, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6326 16-core CPUs, 2 480GB SSDs in software raid 1

Current setup

The dumps monitor which cleans up stale locks and generates the index.html file, currently runs on snapshot1010.

Worker nodes:

En wikipedia dumps normally run on snapshot1012 via the dump scheduler. Wikidata dumps run on snapshot1011 via the scheduler. All other xml/sql dumps normally run on snapshot1010, via the dump scheduler. Dumps other than the xml/sql dumps run on snapshot1008.

The testbed hosts are used for catchup, emergency testing of bug fixes, at scale tests of misc cron jobs, and so on.

For info on running the dumps, see Dumps#Starting_dump_runs.

Other tasks

All misc cron jobs apart from the sql/xml dump runs take place on snapshot1008. From here adds-changes dumps are run, wikidata weekly entity dumps are generated, and so on.