Portal:Toolforge/Admin/Packaging

This page contains information on the various packaging situations we do related to Toolforge. This refers specifically to Debian packages (.deb).

Packages

Approximate list of .deb packages we handle for Toolforge.

toolforge cli

Code browser: https://gerrit.wikimedia.org/r/plugins/gitiles/cloud/toolforge/toolforge-cli/+/refs/heads/main

tools-webservice

When this package is updated, all docker images need to be rebuilt.

jobutils/misctools

gerrit page: https://gerrit.wikimedia.org/r/admin/projects/labs/toollabs
clone URL: https://gerrit.wikimedia.org/r/labs/toollabs
gitiles: https://gerrit.wikimedia.org/g/labs/toollabs

jobs-framework-cli

toolforge build service/pack

code browser: https://gitlab.wikimedia.org/repos/cloud/toolforge/packaging-pack/

Uploading a package

If you built a package or have a package already built that you want to upload, you can scp it to tools-services-* VM, and from there add it to the repository you want it in:

dcaro@vulcanus$ scp pack-0.27.0-2.deb tools-services-05.tools.eqiad1.wikimedia.cloud:/tmp
dcaro@vulcanus$ wm-ssh tools-services-05.tools.eqiad1.wikimedia.cloud
...
# check which repositories you want to add the package to
root@tools-services-05:~# aptly repo list
...
# Add the package to the repo you want
root@tools-services-05:~# aptly repo add buster-toolsbeta /tmp/pack-0.27.0-2.deb
...
# Copy that same package to any other repo if you want
root@tools-services-05:~# aptly repo copy buster-toolsbeta buster-tools pack_0.27.0-1_all
...
# Publish the package on each of the repos you added it
root@tools-services-05:~# aptly publish update --skip-signing bullseye-toolsbeta
...

wmcs-package-build

This is a custom script to automate most of the build & publish steps of the process.

The steps are done in order:

git stage: create a random directory in the package build server, git clone the source repository and checkout a given branch.
sbuild stage: run sbuild to build the debian package.
copy stage: copy the resulting .deb packages from the package build server into the aptly server at a random directory.
aptly stage: add the .deb packages to a given repo and publish it.
backup stage: run rsync from the aptly tree to NFS (optional).
cleanup stage: delete random directories created by the script.

You can locate this script in the puppet tree: modules/toolforge/files/wmcs-package-build.py.

user@laptop:~$ modules/toolforge/files/wmcs-package-build.py --help
usage: wmcs-package-build.py [-h] --git-repo GIT_REPO [--git-branch GIT_BRANCH]
                             [--build-dist BUILD_DIST] [--build-host BUILD_HOST] [-a APTLY_DIST]
                             [--aptly-host APTLY_HOST] [-b] [-d]

Utility to build and upload a .deb package to aptly

optional arguments:
  -h, --help            show this help message and exit
  --git-repo GIT_REPO   git repository URL with the source pkg. This script will do a fresh git clone
                        of that repo. Typical value is something like:
                        https://gerrit.wikimedia.org/r/operations/software/tools-webservice
  --git-branch GIT_BRANCH
                        git branch to use to build the package from. Defaults to "master"
  --build-dist BUILD_DIST
                        target distribution when building the package with sbuild. Defaults to
                        "stretch"
  --build-host BUILD_HOST
                        package build host. Typically a VM in CloudVPS with
                        role::wmcs::toolforge::package_builder. Defaults to "tools-package-
                        builder-02.tools.eqiad.wmflabs"
  -a APTLY_DIST, --aptly-dist APTLY_DIST
                        target distribution in aptly. The resulting deb package will be uploaded to
                        this distribution and then the repository will be published. Can be specified
                        multiple times for multiple target distributions. If this argument is not
                        provided, no aptly operations will be done. Example: -a stretch-tools -a
                        stretch-toolsbeta
  --aptly-host APTLY_HOST
                        aptly server host. Typically a VM in CloudVPS with
                        role::wmcs::toolforge::services. Defaults to "tools-sge-
                        services-03.tools.eqiad.wmflabs"
  -b, --no-backup       If this option is present, this script won't backup aptly data over NFS
  -d, --dry-run         Dry run: only show what this script would do, but don't do it for real

This script is supposed to run from your laptop, and it will start SSH connections to the different servers involved in building and publishing a deb package.

The --help output should have enough information to know what every option does.

You are encouraged to use the --dry-run first to know what the script would do (the actual SSH commands).

For the toolsbeta project

Try first in the toolsbeta project for testing purposes before releasing in the tools project.

For that, build the package and upload it to the -toolsbeta variants of the aptly repos:

user@laptop:~$ wmcs-package-build.py --git-repo https://gerrit.wikimedia.org/r/operations/software/tools-webservice --aptly-dist buster-toolsbeta --aptly-dist bullseye-toolsbeta 
[..]

Then you can install the package in toolsbeta servers for testing.

For the tools project

Once testing is done and you are confident with a given package version, you can copy the packages to the final destination:

root@tools-services-05:~# aptly repo copy buster-toolsbeta buster-tools toolforge-webservice_0.XX_all && aptly publish --skip-signing update buster-tools
root@tools-services-05:~# aptly repo copy bullseye-toolsbeta bullseye-tools toolforge-webservice_0.XX_all && aptly publish --skip-signing update bullseye-tools
[..]

Previously this page recommended using wmcs-package-build.py again for the -tools distributions, but that sometimes fails if the build .debs are not identical (aptly can't different debs with the exact same name and version but in different repositories).

You can use cumin to push the change out everywhere ahead of the next scheduled puppet runs:

$ ssh tools-cumin-1.tools.eqiad1.wikimedia.cloud
$ sudo cumin "P{R:Package = toolforge-webservice}" "apt-get update"
$ sudo cumin "P{R:Package = toolforge-webservice}" 'DEBIAN_FRONTEND=noninteractive apt-get install -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" toolforge-webservice'

sbuild

The sbuild tool is a simple and powerful tool to build debian packages.

Our packaging building hosts have sbuild ready to use. The first time you try to use sbuild you should add yourself to the sbuild unix group:

user@tools-package-builder-02:~$ sudo sbuild-adduser $LOGNAME
[..]
user@tools-package-builder-02:~$ newgrp sbuild

There should be environments to build packages for the most common distros: jessie, stretch, buster.

You can switch between them easily:

user@tools-package-builder-02:mypackage/$ sbuild -d jessie
[..]
user@tools-package-builder-02:mypackage/$ sbuild -d stretch
[..]
user@tools-package-builder-02:mypackage/$ sbuild -d buster
[..]

Sometimes the build-deps required to clean the source tree are not installed outside the chroot. To workardound this, use the --no-clean-source switch when calling sbuild. Adding extra repos and packages is super easy, check the --extra-repository option in the sbuild manpage.

Additional upstream documentation: https://wiki.debian.org/sbuild

Packaging good practices

Some good practices, advises and hints about our packaging workflows:

Use gbp dch --git-author --id-length=7 --since=... to generate debian/changelog file. This reads the git log to generate changelog entries. Make changes to d/changelog in a separate commit.

Does the project provide a script for it? Check if there's a utils/bump_version.sh! It will work on any OS that supports docker/podman

03:54 PM ~/Work/wikimedia/toolforge-cli  (main|✔)
dcaro@vulcanus$ ./utils/bump_version.sh
[+] Building 0.2s (12/12) FINISHED
 ...
Updating changelog...
gbp:info: Found tag for topmost changelog version '66b8b8e47a061e19d743cff9cba1880d651e6dac'
gbp:info: Continuing from commit '66b8b8e47a061e19d743cff9cba1880d651e6dac'
Now you can send this patch for review, remember to create a tag named 'debian/0.2.9' and pushing it when publishing the package.

03:54 PM ~/Work/wikimedia/toolforge-cli  (main|✚ 2)
dcaro@vulcanus$ git diff
diff --git a/debian/changelog b/debian/changelog
index 938eda2..fc636da 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,22 @@
+toolforge-cli (0.2.9) buster; urgency=medium
+
+  [ David Caro ]
+  * [ad4a85e] config: add a config module to load defaults from files
+
+  [ Slavina Stefanova ]
+  * [1205c6f] cli: typo fixes
+
+  [ Raymond Ndibe ]
+  * [75ed92b] toolforge-cli: improve error handling of _execute_k8s_client_method
+  * [2071ec8] toolforge-cli.build.show: less details when --verbose is false
+  * [74bb22a] toolforge-cli: seperate debug from verbose
+
+  [ David Caro ]
+  * [9b7f47c] tox: don't fail if the keyring is not loaded on linux
+  * [793840b] Allow for relative paths to kubeconfig certs
+
+ -- David Caro <dcaro@wikimedia.org>  Thu, 27 Apr 2023 13:54:40 +0000
+
 toolforge-cli (0.2.8) buster; urgency=medium

   [ David Caro ]
diff --git a/pyproject.toml b/pyproject.toml
index 5b208de..b415025 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "toolforge-cli"
-version = "0.2.8"
+version = "0.2.9"
 description = "Toolforge client"
 authors = ["David Caro <dcaro@wikimedia.org>"]
 license = "GPLv3"

Not on a Debian host? Try using Docker!

docker run -it --rm \
  -v "$(pwd):/mnt" -v "$HOME/.gitconfig:/etc/skel/.gitconfig:ro" \
  -e UID=$(id -u) -e GID=$(id -g) \
  opxhub/gbp:buster \
  bash -c 'gbp dch --git-author --id-length=7'

The Docker trick can also be included in a bash init script as a function for your shell so you only need to run gbp and then the rest of the command exactly like on a debian host (even works on an M1 mac so far)

function gbp() {
  docker run -it --rm -v "$(pwd):/mnt" \
    -v "$HOME/.gitconfig:/etc/skel/.gitconfig:ro" -e UID="$(id -u)" \
    -e GID="$(id -g)" opxhub/gbp:buster bash -c "gbp $*"
}

when a new package is released (i.e, uploaded to a repository and/or distributed to clients) a git tag should be created in the git repository with the format debian/version and pushed to Gerrit with git push origin {tagname}
- optionally you can create an annotated tag (git tag -a tagname) and add a short annotation, e.g. the output of git shortlog {previoustag}..HEAD
- tags can also be created from the Gerrit UI (from the Tags link in the sidebar, e.g. https://gerrit.wikimedia.org/r/admin/repos/operations/software/tools-webservice,tags)

Portal:Toolforge/Admin/Packaging

Contents