APT repository
Wikimedia maintains its own APT repository available at https://apt.wikimedia.org/wikimedia/ and currently hosted on apt1001.
This repository contains Debian and Ubuntu packages modified for use by Wikimedia as well as packages for Wikimedia-originated projects.
Repository Structure
Starting with Debian 9 Stretch, we are using a new repository structure with the following components:
main
contains all the packages that we either create internally or packages that we backport or modify from Debian and which are useful fleet-wide (e.g., Icinga plugins, Cumin, etc.)thirdparty/hwraid
contains binary-only drivers for RAID management used on baremetal servers only.- All other components using the
thirdparty/
prefix are synchronised from external repositories (e.g.,thirdparty/cloudera
,thirdparty/confluent
,thirdparty/ci
(Jenkins) andthirdparty/k8s
(Docker)). Please do not add internally built packages tothirdparty
. - Some packages should not be available fleet-wide, but rather be used in more specific scenarios such as the following:
component/ci
provides co-installable PHP packages that are only necessary for CI- "contain" specific packages that depend on outdated or backported libraries (e.g. crypto libraries)
- ease migrations and upgrades for certain roles, while leaving the rest of the fleet untouched
You can find a complete list of repository components in our Puppet config
Older Distributions
For older Linux distributions like Debian 8 Jessie or Ubuntu 14.04 Trusty, the new repository organization was not applied retroactively due to the extra effort required.
In such cases, internally built packages could be in main
or backports
. If you add new packages, always use main
.
Additionally, third-party packages are not fully split into logical components and they could be in the thirdparty
top-level component as well as in sub-component thirdparty/foo
.
If you are updating third-party packages, remember to update either thirdparty
or the thirdparty/foo
sub-component depending on where they were originally located. If you can, a better long-term plan is to update to Debian 9 Stretch.
Finally, Ubuntu 14.04 Trusty also has universe
and non-free
. Ignore those and use main
.
External Access
For Wikimedia servers and Cloud VPS instances, the repositories are automatically configured via Puppet.
To use this repository from an external host, the following lines need to be present in /etc/apt/sources.list
or /etc/apt/sources.list.d/wikimedia.list
:
## Wikimedia APT repository deb http://apt.wikimedia.org/wikimedia stretch-wikimedia main deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia main
Use the right distribution, depending on which Debian/Ubuntu version was installed (e.g. bullseye-wikimedia
, buster-wikimedia
or trusty-wikimedia
).
Here is a complete list of all the available sources:
deb http://apt.wikimedia.org/wikimedia trusty-wikimedia main experimental backports deb http://apt.wikimedia.org/wikimedia jessie-wikimedia main experimental backports deb http://apt.wikimedia.org/wikimedia stretch-wikimedia main deb http://apt.wikimedia.org/wikimedia buster-wikimedia main deb http://apt.wikimedia.org/wikimedia bullseye-wikimedia main deb-src http://apt.wikimedia.org/wikimedia jessie-wikimedia main backports deb-src http://apt.wikimedia.org/wikimedia trusty-wikimedia main deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia main deb-src http://apt.wikimedia.org/wikimedia buster-wikimedia main deb-src http://apt.wikimedia.org/wikimedia bullseye-wikimedia main
Additionally, to make sure that the system prefers packages from this repository, and not packages from the origin distributions even if they have a higher version number. For example, this APT source is "pinned" with a higher priority in /etc/apt/preferences.d/wikimedia.pref
:
Package: * Pin: release o=Wikimedia Pin-Priority: 1001
You can confirm Wikimedia's APT repository is taking preference now by running `apt-cache policy <package>`:
$ apt-cache policy puppetmaster puppetmaster: Installed: (none) Candidate: 4.8.2-5 Version table: 4.8.2-5 1001 500 http://deb.debian.org/debian stretch/main amd64 Packages 1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/main amd64 Packages
Security
The Wikimedia repository is signed with the Wikimedia Archive Automatic Signing Key <root@wikimedia.org>
. This public key must be installed in APT's GPG keyring so it can verify packages successfully.
The key can be retrieved from /Stretch-Key (this key was introduced with stretch and also applies to later distros)
To install it, download the key to /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc
, or on older systems add it using `apt-key`:
# Debian buster or later
$ wget -O /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw"
# Debian Stretch or earlier
$ wget -O wikimedia-apt-key "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw"
$ apt-key add wikimedia-apt-key
OK
You can also retrieve the key from a Cloud VPS host: ssh <host> apt-key export root@wikimedia.org
See also
- pbuilder for building packages for different distribution versions and architectures
- reprepro for additional information on how to use this tool
- Wikimedia APT browser for a simple web package listing