Toolsadmin.wikimedia.org/Build

From Wikitech
Jump to navigation Jump to search

The Striker build and deploy process involves updating the labs-striker-deploy git repository and then pushing the repository out to servers running Striker via scap3.

Deploy directory structure

/srv/deployment/striker/deploy/
 ├── _build/                    # Generated by make_wheels.sh, only on Cloud VPS staging server
 |    └── venv/                 # virtualenv used for creating wheels & top level 
 ├── ddl/                       # Manually created SQL patches
 ├── make_wheels.sh             # Build script for preparing labs-striker-wheels patches
 ├── public_html/               # Apache/nginx docroot
 |    └── static/               # labs-striker-staticfiles submodule
 ├── requirements.txt           # `pip --freeze` generated requirements
 ├── scap/
 |    ├── checks/
 |    |    └──  virtualenv.sh   # Check script to rebuild venv on target hosts
 |    ├── checks.yaml           # Post-deploy checks for Scap3
 |    ├── dsh/                  # Scap3 target host sets
 |    ├── log/                  # Generated by scap3, only on deploy master
 |    └── scap.cfg              # Scap3 config file
 ├── striker/                   # striker submodule
 |    └── requirements.txt      # Product level requirements
 └── wheels/                    # labs-striker-wheels submodule

Submodules

Most of the interesting code changes will happen in submodules of labs-striker-deploy:

labs-striker

There is nothing special to do to prepare this repo for deployment. Merge commits using Gerrit.

Changes made here however will trigger changes to other submodules. The the point a release is being made:

  • labs-striker-staticfiles must be updated if any changes were made to requirements.txt or if files were modified in the static directory.
  • labs-striker-wheels must be updated if any changes were made to requirements.txt.

labs-striker-staticfiles

These static assets are collected from the installed Django applications using Django's collectstatic management script. The contrib/collectstatic.sh convenience script is provided in labs-striker for collecting the files. This can be run in a MediaWiki-Vagrant development environment.

$ cd /vagrant/srv/striker
$ contrib/collectstatic.sh
Deleting 'robots.f71d20196d4c.txt'
Deleting 'robots.txt'
Deleting 'staticfiles.json'
...
Post-processed 'robots.txt' as 'robots.f71d20196d4c.txt'

110 static files copied to '/vagrant/srv/striker/staticfiles', 110 post-processed.
$ cd staticfiles
$ python -mjson.tool staticfiles.json > staticfiles.json.pretty
$ mv staticfiles.json.pretty staticfiles.json
$ cp -R . $PATH_TO_LAB_STRIKER_STATICFILES
$ cd $PATH_TO_LAB_STRIKER_STATICFILES
$ git status
$ git add .
$ git commit
$ git review

Before submitting you may want to look for old assets that can be removed as well. These would generally be <FILENAME>.<CONTENT_HASH>.<EXT> files that are for versions of the files that were removed from the manifest in a prior update that has already been deployed.

labs-striker-wheels

This repo is the collection of Python wheels to be installed in the virtualenv that scap3 manages for each deploy target. We do not allow direct interaction with pypi from inside the production network, so instead we create a collection of wheels outside of prod and save them in gerrit for use inside the prod network. This is not pretty, but it works.

:# Clone the striker/deploy.git repo
$ git clone ssh://gerrit/labs/striker/deploy striker-deploy
$ cd striker-deploy
:# Update the striker submodule checkout to the HEAD of master (or whatever you are prepping for deploy)
$ cd striker
$ git fetch
$ git reset --hard origin/master
:# Remove old wheels locally
$ cd ../wheels
$ git checkout master
$ rm *.whl
:# Re-build wheels cache from scratch
$ cd ..
$ rm -rf _build
$ ./make_wheels.sh
:# Turn the new wheels into a patch for gerrit review
$ cd wheels
$ git add .
$ git commit
$ git review

Testing in Cloud VPS project

TODO: explain how I typically stage pending changes in the striker Cloud VPS project.

$ ssh striker-deploy04.striker.eqiad.wmflabs
$ cd /srv/deployment/striker/deploy/striker
$ git fetch "https://gerrit.wikimedia.org/r/labs/striker" refs/changes/74/521774/1 && git cherry-pick FETCH_HEAD
$ scap deploy

Deploy

This section describes the process of getting the live service modified with new changes.

  • download the source code git repository: https://gerrit.wikimedia.org/r/#/admin/projects/labs/striker
  • download the deployment git repository: https://gerrit.wikimedia.org/r/#/admin/projects/labs/striker/deploy
  • develop: make your code changes, tests, etc (out of the scope of this section)
  • git review the source code (gerrit) and get it merged
  • update the submodule commit in the deployment repo pointing to the commit you want to deploy
    • https://gerrit.wikimedia.org/r/#/admin/projects/labs/striker is a submodule in the deploy repo
    • Occasionally this will not fetch no matter what you do, so I found that running
      git submodule update --recursive --init
      captured the latest code, allowing one to cd into the striker subdirectory and run
      git checkout <desired commit>
      which seems to be how this is often updated.
    • Add, commit and submit your changes for review on the deploy repo. This is the repo you are deploying!
  • ssh jump to deployment server, for example deploy1001.eqiad.wmnet and cd to /srv/deployment/stricker/deploy
  • git fetch & merge/rebase & git submodule update --recursive
    • The submodule won't update unless you changed it on the deploy repository in the last step above
  • scap deploy