Release Engineering/Drafts/Deployments/How to pair on the train

From Wikitech
FIXME: Contextualize, link to canonical train deployments page, navigation menu.

Background

Until October of 2019, running the deployment train was done by one assigned person in two-week shifts. In an effort to better share the workload and improve continuity of knowledge between successive deployers, Release Engineering then moved to a staggered pairing rotation. Each train has a primary conductor and a backup conductor. The following week, the primary moves to backup and a new primary is assigned.

Helpful practices

On Monday, begin by communicating with your partner and establishing how you'll collaborate over the course of the week.

If you are going to be away from your computer during or soon after deploy windows, let your partner know and coordinate coverage for those times. For example, if you have an appointment or family commitments after the deploy on Wednesday, ask your backup conductor to watch logs and the blocker task during that time.

These are rough guidelines for sharing the work, and should be improved as we learn more.

  • Updates on IRC while your partner is working and updates on the train blocker ticket if they're offline seems to be a useful pattern.
  • In general, communicate decisions on the blocker ticket and make use of the Server Admin Log.
  • Liberal use of video chat for pairing on hard problems is encouraged.
  • It works well to have the primary do the work of cutting the branch, syncing wikis, etc., while the backup keeps an eye on logs, works on improvements to deploy tooling, and is generally an extra pair of eyes for the whole process.
  • If you are in doubt about anything and it's during your partner's working hours, consult them first and get their help in resolving your questions.
  • If one member of the pair is in the European window and one is in the American window, both train deployment windows should be reserved on the Deployments calendar. This gives a backup deployer a defined window for moving the train forward outside the primary's working hours, if it becomes necessary.
  • If the train is blocked or there are any other issues, communicate the transfer of responsibility on the train blocker ticket by assigning it to the responsible party and leaving a note.