Dumps/Snapshot hosts
Snapshot (XML dumps generation) cluster information
Hardware
These hosts generate the sql/XML dumps or the "misc cron" dumps. For information about the hosts that serve them, see Dumps/Dump servers.
We have one mini snapshot cluster. I would love it if another were to be set up in codfw. Someday...
In D.C.:
- snapshot1008: primary (other dumps), Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6130 16-core CPUs, 2 480GB SSDs in software raid 1 -- decomm in 2023
- snapshot1009: testbed, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6130 16-core CPUs, 2 480GB SSDs in software raid 1 -- decomm in 2023
- snapshot1010: primary and monitor, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
- snapshot1011: primary, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
- snapshot1012: primary, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
- snapshot1013: primary, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
- snpashot1014: spare, Dell PowerEdge R440, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
- snapshot1015: spare, Dell PowerEdge R440, Debian 10 (buster), 64GB RAM, 2 Intel Xeon Gold 6240 18-core CPUs, 2 480GB SSDs in software raid 1
- snapshot1016: spare, Dell PowerEdge R650xs, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6326 16-core CPUs, 2 480GB SSDs in software raid 1
- snapshot1017: spare, Dell PowerEdge R650xs, Debian 11 (bullseye), 64GB RAM, 2 Intel Xeon Gold 6326 16-core CPUs, 2 480GB SSDs in software raid 1
Current setup
The dumps monitor which cleans up stale locks and generates the index.html file, currently runs on snapshot1010.
Worker nodes:
En wikipedia dumps normally run on snapshot1012 via the dump scheduler. Wikidata dumps run on snapshot1011 via the scheduler. All other xml/sql dumps normally run on snapshot1009 and snapshot1010, via the dump scheduler. Dumps other than the xml/sql dumps run on snapshot1008.
The testbed host is used for catchup, emergency testing of bug fixes, at scale tests of misc cron jobs, and so on.
For info on running the dumps, see Dumps#Starting_dump_runs.
Other tasks
All misc cron jobs apart from the sql/xml dump runs take place on snapshot1008. From here adds-changes dumps are run, wikidata weekly entity dumps are generated, and so on.