Phabricator/Disaster Recovery

For reference, a list of patches required to migrate to a new Phabricator server: https://gerrit.wikimedia.org/r/q/topic:phab1002
List of patches for migration from phab1001 to phab1003: https://gerrit.wikimedia.org/r/q/%2523phabricator-server-switch

Switch Phabricator production to phab2002 (in codfw)
- verify that git-ssh is working on phab2002
- shut down phd on phab1004
- manually rsync the git data from phab1004 to phab2002 (this is already rsync'd periodically, just need a one-time refresh)
- verify that phd works on phab2002
- switch `phabricator_active_server` to `phab2002` in hiera, `role/common/phabricator/main.yaml`
- test phabricator's web interface by locally overriding dns records to point to phab2002
update dns / redirect traffic to phab2002
- Update SFP records for the phabricator hostname (example)
Make phab1004 a warm standby for phab2002
- verifying that everything is installed correctly
- manually rsync the git repositories.
- make sure the rsync cron job is set up correctly to sync from phab2002