Kubernetes/Packages

Intro

For a variety of reasons, we deploy our kubernetes components in WMF production (Tools/Toolforge is a completely different environment) using Debian packages. Those are:

kubernetes-client one has kubectl in it, kubernetes-master has kube-apiserver, kube-scheduler, kube-controller-manager and kuberetes-node has kubelet and kube-proxy components.

Packaging

We usually build on bullseye and copy the resulting packages to other distros as needed.

We don't actually build kubernetes but package it's components from upstream binary releases as described above.

Part of the process is to download the release tarball and verify its sha512 hash against the one found in the current git master CHANGELOG.

Because of that, you will need to set HTTP proxy variables for internet access on the build host.

The general process to follow is:

Check out operations/debs/kubernetes on your workstation
Switch to the major version branch if it already exists or create a new one (vX.Y)
Create a patch to bump the debian changelog

export NEW_VERSION=1.19.3 # Kubernetes version you want to packae
dch -v ${NEW_VERSION}-1 -D bullseye-wikimedia "Update to v${NEW_VERSION}"
git commit debian/changelog

# Make sure to submit the patch to the correct branch
git review vX.Y

Merge

Check out operations/debs/kubernetes on the build host
Build the packages:

git checkout vX.Y

# Ensure you allow networking in pbuilder
# This option needs to be in the file, an environment variable will *not* work!
echo "USENETWORK=yes" >> ~/.pbuilderrc

# Build the package
https_proxy=http://webproxy.$(hostname -d):8080 DIST=bullseye pdebuild

Publishing

# On apt1001, copy the packages from the build host
rsync -vaz build2001.codfw.wmnet::pbuilder-result/bullseye-amd64/kubernetes*<PACKAGE VERSION>* .

# Import packages to the corresponding kubernetes apt component
sudo -i reprepro -C component/kubernetesXY --ignore=wrongdistribution include bullseye-wikimedia /path/to/<PACKAGE>.changes

# Kubernetes packages can be copied between distros:
sudo -i reprepro -C component/kubernetesXY copysrc buster-wikimedia bullseye-wikimedia kubernetes
sudo -i reprepro -C component/kubernetesXY copysrc bookworm-wikimedia bullseye-wikimedia kubernetes