Jump to content

Help:Development recommendations for easily moving to production

From Wikitech

This page describes processes for advanced users, specifically individuals who are developing services aimed at Wikimedia production.

Avoid pip, pecl, gems, and other language specific package managers

Please, take care when using language specific repositories, like pip, pecl, etc. In production we do not use language specific package managers because they generally do not provide cryptographic end-to-end trust guarantees for the software they install.

When pushing to production, all dependencies must:

  • Be installable as Debian packages
  • From the official Debian repository or Wikimedia hosted apt repositories

If a package isn't available in the official Debian repository can be packaged and hosted in a custom Wikimedia repository.

Language specific package managers

When using language specific repositories, newer versions of libraries than are available in the Debian repository may be installed. Your application may then depend on the newer versions, and when we puppetize the system, the newer versions have to be backported to whichever distro version we are using.

We recommend that before using a language specific package manager, you check to see if the required library exists in the Debian repository first. Unless the packaged version is too old, please use it rather than the version in the language specific package manager. When using a language specific package manager, please document the library that was installed, so that it can be packaged when the service is puppetized and deployed to production.

Document all package installations, all modified configuration files, and any commands run to configure a service

Documenting how a service is installed and configured makes the puppetization of a service much easier. It should be possible to fully puppetize a system just by reading the documentation. This also means that when a service is changed after being deployed to production that its documentation should be updated as well.

Know and document any ports and protocols in use by your service

It's important to know which ports and protocols are in use by your service, and how that affects the security of a service. Memcache, for instance, runs on 11211, and by default has no security. It's important to firewall the service, or to ensure it isn't accessible to the public. Many services have considerations like this, and documenting these makes them much easier to puppetize.

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support
Stay aware of critical changes and plans
Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)