Data Platform Engineering/Share our work

When working remotely as a team (we are all remoties!), it is especially important to 'share our work' in writing. At the Wikimedia Foundation, we have several channels for asynchronous collaboration: Phabricator, code review (gerrit or gitlab), Slack, IRC, email, mailing lists (with public archives) wikis (mediawiki.org, wikitech.wikimedia.org, meta.wikimedia.org), Miro boards, Asana, Google docs, calls and meetings, etc. It can sometimes be difficult to follow all of these channels, and it can be even more difficult to find past work and decisions made.

This page describes how we in Data Platform Engineering (and perhaps more broadly for WMF Engineering) can use these channels to effectively collaborate with each other.

tl;dr

Sharing and documenting our work in writing is part of doing the work.
Phabricator and Wikis are the only valid place to document our work.
Google docs, Slack, IRC, email, Miro boards, Asana, meetings and code review are for collaboration, not for documentation.

Why

Sharing our work in writing...

helps us collaborate more effectively and efficiently as a remote first organization.
allows us to discover and understand each other's work
allows others to join the conversation
allows for a work to be handed over to someone else
in Phabricator and wikis is discoverable via search
highlights scope of work for current and future planning
shows how close the task is to completion for planning purposes
helps us to avoid repeating mistakes

Where

Wikis

wikitech.wikimedia.org and mediawiki.org are the canonical places for technical documentation. All final documentation should be on a wiki. These wiki docs can link to Phabricator tasks for folks who want to see the context around how decisions were made and solutions implemented.

Phabricator

Phabricator should be used for project and task tracking, as well as the main place to document our day to day work. When working on a Phabricator task, we should use it as a place to post status updates of how the task is going, as well as a discussion log between people working on the task.

Because Phabricator conversations are public and available forever, it allows people in the future to find the context of how and why certain work was done.

For project work, avoid creating 'island' Phabriactor tasks. Small tasks can be good for tracking and planning purposes, but you should always link them back to parent tasks. This allows others to understand the context of the work when they encounter a task.

Quick bug reports and one offs may not need linking to parent tasks, but please add appropriate team or project tags, and subscribe appropriate people if you can.

Try to keep high level conversations and decisions in related larger parent tasks. Conversations in small tracking tasks tends to create fragmented conversations and context.

Code review

Code review should be used for directly reviewing and collaborating on code that we commit. Sometimes, conversations in code review also end up being the kind of conversations someone not involved in the code review might want to follow or find in the future. When this happens, a summary of the conversation should be posted on a Phabricator task.

Meetings, chat, and email

Realtime collaboration is necessary, but conversations in meetings and Slack and IRC rooms are siloed and difficult to find for posterity. Summaries of important conversations and decisions made should be posted on a Phabricator task or a wiki page.

Avoid direct messages for discussing and deciding on work. Direct messages are useful when you want to discuss sensitive things, or perhaps early vetting of an idea. Because direct messages are private, using them to make decisions hides the context and complexity involved in making a decision, and removes the opportunity for someone with more context to weigh in. Instead, use a relevant public IRC room, a shared Slack workspace, or even better a Phabricator ticket.

Google docs

Google docs can be used for design documents and project planning, but ultimately the final status of design documents should be put on Phabricator or a wiki. Once moved, the Google doc should have header with a note and link to where the final doc has been migrated. The final doc can link back to the Google doc as well.

Example

Let's take a real example of a Phabricator task where the folks involved shared their work well.

T321088 - Add support for jupyterhub on conda-analytics

This task's description of work is clear. This clear description made it possible for concerned users to ask questions.
The code review has conversations about the code itself, but those conversations don't reflect anything meaningful to the task in question.
After some code review has happened, Xabriel summarizes the status and next steps. This allows anyone following along to quickly understand.
Xabriel documents a testing plan that he discussed elsewhere with Ben. there was realtime back and forth to draft this plan, but the final version was posted to Phabricator. This might allow a future engineer to find out how this had been done in the past, and repeat it.
While deploying, a problem was encountered. The problem is described and a workaround is noted. Ben later fixes the problem and describes how.
Another user following along was able to refer to a past discussion in a different Phabricator ticket, in order to make an improvement to this work. If this past discussion was not in Phabricator, it would have been harder to summarize and link to the context.
Ben and Xabriel continue to collaborate and document their work on Phabricator. The next time this kind of work needs to be done, or if someone needs to figure out how and why this was deployed in this way, they will have the information they need.