Maps/Services deployment

Deploying maps services

Before you actually start deploying check out some basic tips for deployment.

Developer setup for updating deployment repo

The deployment repo is used to compose the kartotherian and tilerator code and it's dependencies. Before a new versions can be deployed there must be an update to the deployment repo.

We build the deploy repo in Docker in order to ensure that the Node modules have the correct binaries. We build the deploy repo using Docker for Mac or a Linux machine. Before you start the first build of the deploy repo run through the setup instructions.

Kartotherian is developed on Gerrit, and the old Github repository is abandoned and is no longer used by WMF infrastructure.

Publish library dependencies to NPM before building for deployment

In order to be able to package kartotherian and tilerator, the process to bump the version and publish to npm is crucial, therefore you should have permission to publish packages in the @wikimedia npm organization. The version bump occurs using manually package by package or using the lerna dependency manager.

Building for deployment

Kartotherian and Tilerator are deployed according to the standard process for deploying Wikimedia Node.js services.

Update and build the deployment repo

Sync the code and deploy repos with current master.

cd ~/path/to/kartotherian/deploy
git status
git reset --hard origin/master
git checkout .
git clean -fd
git checkout master
git pull
git --no-pager log --decorate -n 1
git submodule update --init
cd ./src
git checkout master
git pull
cd ..
git add src
git branch
git status

At this point, commit any updates to the src submodule.

If using Docker for Mac start the Docker daemon by clicking on the Whale icon in the menu bar. (Should work automatically on Linux.) Run the tests in Docker and build the new commit for the deploy repo:

./server.js build --deploy-repo --force

And push to Gerrit:

git review

Deploy to Beta Cluster

The steps are similar to #Deploy to Production but using different machines, of course.

To deploy:

ssh deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud

scap deploy --environment beta "`git log --pretty=format:'%s' -n 1`"

To verify service deployment you can ssh into deployment-maps-master01.deployment-prep.eqiad1.wikimedia.cloud.

Use #wikimedia-releng (instead of #wikimedia-operations) to see if there are issues. You may want to log manually in this channel using !log until phab:T156079 is resolved. This logs to the Releng team's Server admin log

Deploy to Production

Scan through recent chat in #wikimedia-operations channel on IRC to make sure there's nothing blocking the deploy.

Optional: Look at deployment logs:

ssh deployment  #deployment.eqiad.wmnet, currently points to deploy1002
cd /srv/deployment/kartotherian/deploy/
scap deploy-log

In another terminal start the actual deployment:

ssh deployment
cd /srv/deployment/kartotherian/deploy/
git pull && git submodule update
git log -n 1
scap deploy "`git log --pretty=format:'%s' -n 1`"

The scap deploy command above takes a reason string argument. If this string contains phab tasks, those tasks will get comments about the deployment happening (start + finish). So, let's say in the deployment we have fixes for tasks T123 and T234 you could write instead of the last command:

scap deploy "`git log --pretty=format:'%s' -n 1` (T123 T234)"

Do the same for Tilerator

Deploying services

Refer to Services/Deployment for general instructions.

In most cases, Kartotherian and Tilerator should be deployed together, to ensure that all dependencies (in particular, styles) are in sync.

Monitoring

To watch logs on the maps server, a custom script has been included:

tail-kartotherian -f