Deploying maps services
Developer setup for updating deployment repo
Check out the basics for deployment.
We build the deploy repo in Docker in order to ensure that the Node modules have the correct binaries. We build the deploy repo using Docker for Mac or a Linux machine. Before you start the first build of the deploy repo run through the instructions.
Publish libraries to NPM before building for deployment
In order to be able to package kartotherian and tilerator, the process to bump the version and publish to npm is crucial, therefore you should have permission to publish packages in the @wikimedia npm organization. The version bump occurs using manually package by package or using the lerna dependency manager.
Building for deployment
Kartotherian and Tilerator are deployed according to the standard process for deploying Wikimedia Node.js services, with the important difference that deployments are built from purpose-specific "package" repos rather than directly from service code repos. This is to facilitate the bundling of additional maps-specific dependencies.
Update and build the deployment repo
Sync the code and deploy repos with current master:
cd ~/path/to/kartotherian/package git status git reset --hard origin/master git checkout . git clean -fd git checkout master git pull git status git --no-pager log --decorate -n 1 rm -rf node_modules/ npm i service-runner cd ../deploy git status git reset --hard origin/master git checkout . git clean -fd git checkout master git pull git --no-pager log --decorate -n 1 git submodule update --init git branch git status cd ../package
If using Docker for Mac start the Docker daemon by clicking on the Whale icon in the menu bar. (Should work automatically on Linux.) Run the tests in Docker and build the new commit for the deploy repo:
./server.js build --deploy-repo --force
And push to Gerrit:
cd ../deploy git review
Deploy to Beta Cluster
The steps are similar to #Deploy to Production but using different machines, of course.
(instead of instead of ssh deployment.)
To verify something on the box you can ssh into
#wikimedia-releng (instead of
#wikimedia-operations) to see if there are issues. You may want to log manually in this channel using
!log until phab:T156079 is resolved. This logs to the Releng team's Server admin log
Deploy to Production
Scan through recent chat in
#wikimedia-operations channel on IRC to make sure there's nothing blocking the deploy.
Optional: Look at deployment logs:
ssh deployment #deployment.eqiad.wmnet, currently points to deploy1002 cd /srv/deployment/kartotherian/deploy/ scap deploy-log
In another terminal start the actual deployment:
ssh deployment cd /srv/deployment/kartotherian/deploy/ git pull && git submodule update git log -n 1 scap deploy "`git log --pretty=format:'%s' -n 1`"
The scap deploy command above takes a reason string argument. If this string contains phab tasks, those tasks will get comments about the deployment happening (start + finish). So, let's say in the deployment we have fixes for tasks T123 and T234 you could write instead of the last command:
scap deploy "`git log --pretty=format:'%s' -n 1` (T123 T234)"
Do the same for Tilerator
Refer to Services/Deployment for general instructions.
In most cases, Kartotherian and Tilerator should be deployed together, to ensure that all dependencies (in particular, styles) are in sync.