Deploying maps services
Before you actually start deploying check out some basic tips for deployment.
Developer setup for updating deployment repo
The deployment repo is used to compose the kartotherian and tilerator code and it's dependencies. Before a new versions can be deployed there must be an update to the deployment repo.
We build the deploy repo in Docker in order to ensure that the Node modules have the correct binaries. We build the deploy repo using Docker for Mac or a Linux machine. Before you start the first build of the deploy repo run through the setup instructions.
Kartotherian is developed on Gerrit, and the old Github repository is abandoned and is no longer used by WMF infrastructure.
Publish library dependencies to NPM before building for deployment
In order to be able to package kartotherian and tilerator, the process to bump the version and publish to npm is crucial, therefore you should have permission to publish packages in the @wikimedia npm organization. The version bump occurs using manually package by package or using the lerna dependency manager.
Building for deployment
Kartotherian and Tilerator are deployed according to the standard process for deploying Wikimedia Node.js services.
Update and build the deployment repo
Sync the code and deploy repos with current master.
cd ~/path/to/kartotherian/deploy git status git reset --hard origin/master git checkout . git clean -fd git checkout master git pull git --no-pager log --decorate -n 1 git submodule update --init cd ./src git checkout master git pull cd .. git add src git branch git status
At this point, commit any updates to the
If using Docker for Mac start the Docker daemon by clicking on the Whale icon in the menu bar. (Should work automatically on Linux.) Run the tests in Docker and build the new commit for the deploy repo:
./server.js build --deploy-repo --force
And push to Gerrit:
Deploy to Beta Cluster
The steps are similar to #Deploy to Production but using different machines, of course.
ssh deployment-deploy03.deployment-prep.eqiad1.wikimedia.cloud scap deploy --environment beta "`git log --pretty=format:'%s' -n 1`"
To verify service deployment you can ssh into
#wikimedia-releng (instead of
#wikimedia-operations) to see if there are issues. You may want to log manually in this channel using
!log until phab:T156079 is resolved. This logs to the Releng team's Server admin log
Deploy to Production
Scan through recent chat in
#wikimedia-operations channel on IRC to make sure there's nothing blocking the deploy.
Optional: Look at deployment logs:
ssh deployment #deployment.eqiad.wmnet, currently points to deploy1002 cd /srv/deployment/kartotherian/deploy/ scap deploy-log
In another terminal start the actual deployment:
ssh deployment cd /srv/deployment/kartotherian/deploy/ git pull && git submodule update git log -n 1 scap deploy "`git log --pretty=format:'%s' -n 1`"
The scap deploy command above takes a reason string argument. If this string contains phab tasks, those tasks will get comments about the deployment happening (start + finish). So, let's say in the deployment we have fixes for tasks T123 and T234 you could write instead of the last command:
scap deploy "`git log --pretty=format:'%s' -n 1` (T123 T234)"
Do the same for Tilerator
Refer to Services/Deployment for general instructions.
In most cases, Kartotherian and Tilerator should be deployed together, to ensure that all dependencies (in particular, styles) are in sync.
To watch logs on the maps server, a custom script has been included: