Obsolete:Backup procedures

This page contains historical information. It is probably no longer true.

THIS PAGE IS OLD AND OUTDATED. It is kept for historical reasons. Most of the info inside will either be or already is wrong/irrelevant. Backup infrastructure has been replaced by Bacula

This page is an inventory of our backups for both clusters data and office data.

Some definitions:

On-site

here means the backup is in the same physical location as the master, but on a different machine.

Off-site

means the backup is in a physically separate location. In most cases it's nice to have both!

Red cells

denote a lacking area which we want/need.

Service backup state

Service	Master	On-site	Off-site	Notes	Approx. Space Needed
Wiki primary data
Wiki main databases	EQIAD db*	EQIAD slave replication, LVM snapshots every 8 hours (have we tested restoring from one?)	PMTPA db slaves & snapshots / ESAMS toolserver replication (how complete is the toolserver repl?)	http://noc.wikimedia.org/dbtree/	4 TB - check
External storage	EQIAD db*	EQIAD slave replication	PMTPA - ? /ESAMS toolserver replication - how complete are these?		5 TB - check
Wiki data dumps	PMTPA Obsolete:dataset2	Gluster storage (labstore1 under /publicdata-project directory) Last 5 good only.	EQIAD: via rolling rsync Obsolete:dataset1001; via daily rsync, all public data: dumps.wikimedia.your.org, last 5 good dumps: wikipedia.c3sl.ufpr.br	See Dumps/Mirror status for more info.	6T (last 5 good)/ 29T (all) - update these
Images and media
Uploads	PMTPA Swift	3 copies per file	EQIAD Ceph		23T?
Thumbs	PMTPA Swift	3 copies per file	EQIAD Ceph		12T?
Software development and configuration
MediaWiki config	PMTPA NFS	tridge	Subversion
Apache config	PMTPA NFS	tridge	Automatic	rsync
Subversion	Formey	tridge			3 GB
Gerrit	Manganese		Formey	Databases on db9/10	3 GB
Bugzilla data	PMTPA db9	db10 replication	EQIAD db1008/db1025 & snapshots		1 GB
Bugzilla config	PMTPA db9	db10 replication	EQIAD db1008/db1025 & snapshots	db10.pmtpa.w /a/sqldata 0 66532680 9020197 13.6 31:49 4724.1 31:49 4724.1	1.5 GB
Bugzilla frontend	PMTPA isidore	daily rsync to Tridge
Wikitech wiki	linode		PMTPA tridge; ESAMS ?
Communications
OTRS data	PMTPA db9	db10 replication	EQIAD db1008/db1025 & snapshots		57 GB
OTRS config	PMTPA bart				19 GB
blogs frontend data	PMTPA singer	Daily backups to Tridge
bugzilla	PMTPA isidore	Daily backups to Tridge
IMAP mail	PMTPA sanger	PMTPA mchenry, Daily backups to tridge		see Mail#Backups	32 GB
Mailing lists	ESAMS lily - decommissioned	EQIAD sodium	PMTPA mchenry	See Mailing lists#Backups	50 GB
DNS	EQIAD ns0, PMTPA ns1	in gerrit	ESAMS ns2	In gerrit
Google docs	Google :P			ad-hoc downloads
Soft data
Search databases	PMTPA various	EQIAD is primary now	SDTPA
HTTP logs	PMTPA locke	emery	/a/squid	oxygen (eqiad)
MediaWiki logs	PMTPA NFS	tridge	fluorine	/home/wikipedia/logs
Fundraising
Fundraising Front-End	EQIAD aluminium		software:svn logs:storage3->tridge	puppet:files/misc/scripts/offhost_backups	5GB
Fundraising Databases	EQIAD db1008	db1025 replication + snapshots	PMTPA storage3 replication, snapshots, dumps tridge daily rsync of dumps	puppet:files/misc/scripts/dump_fundraisingdb	1TB (15GB/day)
Payment Processing	software:svn db+logs:PMTPA	db:mysql replication db+logs:encrypt+copy daily to tridge	db+logs:replicate to silicon	payments cluster	100GB(?)
Impression Logs	PMTPA nas1		nas1001		1.5TB year
Misc data
Server home directories	PMTPA nfs1 / nfs2	tridge		Daily backups to Tridge	30 GB
stat1 /a and /home	-	Amanda to tridge		gerrit change	400 GB