Jump to content

Debian packaging/Package your software as deb

From Wikitech

Introduction

When software we develop at Wikimedia needs to be packaged for Debian, you typically want minimal disruption to your development process. This means that, for instance, maintaining a per-distribution branch where you'll build the software for different Debian versions is inconvenient, if the only thing that has to change is the debian/changelog entry.

Additionally, when developing software, you will typically have project-specific CI pipelines, and any additional pipeline will have to merged into it.

To this end, we've developed a specialized pipeline that builds a Debian package every time that both of the following conditions are true:

  • a commit is pushed or merged onto your main branch
  • there is a change to debian/changelog

The deb package will be built for the distribution indicated in the latest entry in the debian/changelog (which must be in the format $distro-wikimedia, or the CI pipeline will fail).

How to enable automatic build of debian packages

Typically, you have your own CI tasks for your code already. So, just modify your .gitlab-ci.yml file to include the following: 

# you can have more stages, but these are mandatory or CI will fail
stages:
  - prepare
  - build

# Again this just needs to be added to your own set of variables if you have them.
# Please note: this needs to be declared before the include.
variables:
  # Use backports when building debs
  USEBACKPORTS: "1"
  # Do not change this.
  WMF_CI_RELEASE_DEB: "0"

include:
  - project: 'repos/sre/wmf-debci'
    ref: main
    file:
      - 'wmfdeb.yml'

See for examplethis pipeline, where the build_ci_deb job creates the deb packages as artifacts. There is a cronjob on the apt staging repository that scans gitlab's CI to fetch and import the new debs in the repository automatically. Given you want to import your packages into the main apt repository eventually, you need to also do the setup to promote them to the main repository.

A typical workflow

Let's see how a typical workflow will pan out:

Let's say we want to make a new release for vopsbot, and we want to package the software for both Debian Bullseye and Debian Bookworm.

Bookworm

  • First, we'll add an entry to debian/changelog for Bookworm, eg.
 vopsbot (0.3.8-4) bookworm-wikimedia; urgency=medium
 [...]
  • push the change to main
  • The bookworm package will be built:
dch -v 0.3.8-4
git add debian/changelog
git commit -m 'Build 0.3.8-4 for bookworm'
git push
# This triggers a pipeline like https://gitlab.wikimedia.org/repos/sre/vopsbot/-/pipelines/37699

Bullseye

 vopsbot (0.3.8-4~deb11u1) bullseye-wikimedia; urgency=medium
 [...]
  • push the change to the main branch of the repository.
  • The bullseye package will be built:
dch -v '0.3.8-4~deb11u1'
git add debian/changelog
git commit -m 'Build 0.3.8-4 for bullseye'
git push

# This triggers a pipeline like https://gitlab.wikimedia.org/repos/sre/vopsbot/-/pipelines/37702

The important detail is that you need to perform separate push operations for each of the distributions you want to build for, as the pipeline only looks for the latest changelog entry.

An alternative approach, if you have to build for multiple distributions, is to build them all at once by doing tricks with the changelog file in CI, see for example how conftool does it.

Semi-automated upload to apt.wikimedia.org

Once your packages have been built, you should check they've been uploaded to the staging apt repository, then log into the server running the main apt repository and run

$ sudo -i reprepro  --noskipold --restrict vopsbot checkupdate bookworm-wikimedia
Updates needed for 'bookworm-wikimedia|main|amd64':
'vopsbot': '0.3.8-1' will be upgraded to '0.3.9-1+deb12u1' (from 'vopsbot'):
 files needed: pool/main/v/vopsbot/vopsbot_0.3.9-1+deb12u1_amd64.deb
$ sudo -i reprepro --noskipold --restrict vopsbot update bookworm-wikimedia
Calculating packages to get...
Getting packages...
Installing (and possibly deleting) packages...
Exporting indices...
$
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Debian_packaging/Package_your_software_as_deb&oldid=2247196"