Obsolete:Upload filesystem snapshots
Questions
- Do we actually have Zfs snapshots set up on ms7 for upload files?
- How to access past snapshots?
- How the the snapshotting configured? Cronjob? How do we alter it or set it up on a new server?
- How does replication of snapshots to other hosts work? What happens if it breaks?
Answers
On ms4 and on ms7 we have snapshots running. See below for info.
Current setup
Zfs snapshots run out of cron (see /var/spool/cron/crontabs/root) hourly, daily, weekly, monthly and yearly. The command is
/opt/local/bin/zfs-snapshot.images -n weekly
or
/opt/local/bin/zfs-snapshot.thumbs -n weekly
(daily/monthly etc). depending on whether we are dealing with the image or the thumbs server. This is a ksh script which figures out the right name and then runs
zfs snapshot fsname-@frequency-date
Changing the cronjobs
Make your changes in place on the host with crontab -e root and then please put those changes into the file on /home/wikipedia/conf/zfs-tools so we have a copy of it.
Looking at the snapshots
You can see the existing snapshots at any time by
zfs list
You can grab files out of the old upload snapshots by
cd /export/upload/.zfs/snapshot
and go walk through the list of them. Note that ls -a /export/uploads will not show you the .zfs folder. Just be bold and cd into it :-P Similarly go to /export/thumbs/.zfs/snapshot for those.
The parts ofthe crontab that have to do with zfs (snapshots, replication, metrics) are in /home/wikipedia/conf/zfs-tools with a README even, along with the scripts on which they depend.
Setting up on a new server
- Grab the zfs scripts from /home/wikipedia/conf/zfs-tools and put them over onto your new server in /opt/local/bin and /opt/wmf/bin (check the subdirs optlocalbin and (needs updating) opttsbin for those).
- Grab the appropriate lines from one of the crontabs in /home/wikipedia/conf/zfs-tools and add them to your server's crontab.
This assumes you have already set up zfs filesystems and pools appropriately. If you have different names for the filesystem or the pool, you will have to edit the zfs-* and new_file_metric scripts to reflect this.
How the repliaction works, and how to fix it when it breaks
See Zfs replication for all the grody details.