Deployment pipeline/Components

From Wikitech
This page describes the previous deployment pipeline mechanism based on PipelineLib. For information about the current deployment pipeline, which uses GitLab, Blubber, and Kokkuri, see the deployment pipeline documentation on


Blubber creates Dockerfiles from a higher level description expressed as YAML. The Deployment pipeline relies on an application's Blubberfile(s) to build images for testing and production.


PipelineLib is a library of code that consumes an application's .pipeline/config.yaml. This configuration directs the pipeline through integration telling it:

  • What blubber.yaml
  • What unit tests to run
  • What integration tests to run
  • Which tests can be run in parallel
  • What container image is to be used in production


Helm uses Helm charts to define the resources needed to deploy an application to production. Helm manages things like:

  • Config files
  • Resource limits
  • What ports your application uses
  • Smoke tests for your application

In short, Helm allows the easy management of mapping deployment resources to an environment. If you need a different configuration for local development and production, chances are it'll be easy to configure with Helm.


Helmfile is what you will use to deploy your application in production. Helmfile uses the environments and values in the deployment charts repository to define a deployment to a specific environment. A deployment of an existing production service looks like:

$ cd /srv/deployment-charts/helmfile.d/services/blubberoid
$ helmfile -e staging -i apply


Kubernetes is the execution framework that the pipeline works on. See Kubernetes

Application Metrics monitoring

Monitoring in the pipeline happens via prometheus and exposing via Grafana.


Applications need to either provide Prometheus native endpoints (recommended), or emit Statsd (deprecated) compatible metrics to a local prometheus-statsd-exporter instance. In the latter case, SRE provides a ready to use component as part of any Helm chart in the deployment-charts repo.

In all cases, applications are discovered automatically and scraped by Prometheus.


Deployment pipeline applications are currently exposed under the Service folder in See for more