User:Ladsgroup/Shipping dependencies using wheels

Since we are using wheels to ship our dependencies to prod, I thought it would be good to share my experience.

Building wheels and keeping them

In order to install you need to install dependencies recursively otherwise it won't work. Also it is vital to build them in a environment as similar as possible to the production. For example if your nodes will be using jessie, build them in jessie. In order to do that, first build a clean virtualenv (with your preferred python version):

virtualenv -p python3 clean
source clean/bin/activate

then install your dependencies. e.g.:

pip install -r requirements.txt

Then get list of dependencies that got installed:

pip freeze > requirements-frozen.txt

builds wheels for them:

pip install wheels
mkdir wheels/
pip wheels -w wheels/ -r requirements-frozen.txt

It might take a while but once it's done you can see .whl files saved in the wheels folder. Put them in a git repo (like research-ores-deploy-wheels).

Using wheels

While using it's very important to use --no-deps otherwise it can't work. This is an example from ORES:

venv="/srv/ores/venv"
mkdir -p $venv
virtualenv --python python3 --system-site-packages $venv
$venv/bin/pip freeze | xargs $venv/bin/pip uninstall -y
$venv/bin/pip install --use-wheel --no-deps $deploy_dir/submodules/wheels/*.whl