Wikimedia maintains its own APT repository, under http://apt.wikimedia.org/wikimedia which lives on install1002. This repository contains Debian/Ubuntu packages modified for use by Wikimedia and Wikimedia specific native packages, that don't exist elsewhere.
Starting with stretch we are using a new repository structure:
- main contains all the packages that we either create internally or packages that we backport or modify from Debian and which are useful fleet-wide (e.g. Icinga plugins, Cumin etc.)
- thirdparty/hwraid contains binary-only drivers for hardware/RAID management/monitoring, these are only added on baremetal servers
- All other repository components using the thirdparty/ prefix are synchronised from external repositories, e.g. thirdparty/cloudera, thirdparty/confluent, thirdparty/ci (Jenkins) and thirdparty/k8s (docker). A complete list can be found in our puppet config. Please do not add internally built packages to thirdparty.
- Some packages should not be available fleet-wide, but rather be used in more specific scenarios. One example is component/ci, which provides co-installable PHP packages which are only necessary for CI. A complete list can be found in our puppet config.
For our older distros this split was not applied retroactively as it would have taken quite a bit of effort, so there are some differences:
- For jessie and trusty internally built packages are spread out across main and backports. If you add new packages to jessie or trusty, simply always use main.
- For jessie and trusty there's a general thirdparty component, not fully split into logical components (some already exist, e.g. thirdparty//cloudera). If you update an existing deb, update the respective thirdpary/foo components, otherwise put it in thirdparty (or even better migrate to stretch!)
- Trusty also has universe and non-free, simply ignore this and use main.
Accessing the repository externally
For Wikimedia servers and Cloud VPS instances the repositories are automatically configured via puppet.
To use this repository from an external host, the following lines need to be present in /etc/apt/sources.list or /etc/apt/sources.d/wikimedia.list:
## Wikimedia APT repository deb http://apt.wikimedia.org/wikimedia jessie-wikimedia main experimental backports deb-src http://apt.wikimedia.org/wikimedia jessie-wikimedia main universe
Use the right distribution (e.g. trusty-wikimedia) instead of jessie-wikimedia, depending on which Debian/Ubuntu version was installed.
A complete list is
deb http://apt.wikimedia.org/wikimedia trusty-wikimedia main experimental backports deb http://apt.wikimedia.org/wikimedia jessie-wikimedia main experimental backports deb-src http://apt.wikimedia.org/wikimedia jessie-wikimedia main backports deb-src http://apt.wikimedia.org/wikimedia trusty-wikimedia main
Additionally, to make sure that the system prefers packages from this repository, and not packages from the origin distributions even if they have a higher version number, this APT source is "pinned" with a higher priority in /etc/apt/preferences:
Package: * Pin: release o=Wikimedia Pin-Priority: 1001
The Wikimedia repository is signed with the Wikimedia Archive Automatic Signing Key <email@example.com>. This public key must be installed in APT's GPG keyring for it to verify packages without nagging about it. This can be accomplished by installing the wikimedia-keyring package.
- get the key from /Key or for stretch /Stretch-Key
$ wget -O wikimedia-apt-key "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Key&action=raw" $ apt-key add wikimedia-apt-key OK
$ wget -O wikimedia-apt-key "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw" $ apt-key add wikimedia-apt-key OK
- pull the key from a Cloud VPS host:
ssh <host> apt-key export firstname.lastname@example.org, then add with