PipelineLib/Guides/How to configure CI for your project

Welcome to Configure our CI system to execute your pipeline, a simple PipelineLib guide to get your project's newly defined test pipeline running for each patchset submitted to Gerrit.

Prerequisites

• This guide is meant as a supplement to one of the language specific guides. Follow one that fits with your project before continuing here.
• Some knowledge of JJB/Zuul and how to edit integration/config would definitely help during this guide, but you can probably get by with just following the simple steps herein.

Guide

Configure our CI system to execute your pipeline

We're not quite to full self-service CI yet, so you still have to add a little of your own glue to tell our CI system how to invoke your newly defined test pipeline.

Clone integration/config

dev@laptop:~$ git clone ssh://gerrit.wikimedia.org:29418/integration/config src/integration/config
dev@laptop:~$ cd src/integration/config

Tell Jenkins about your project's test pipeline

Define a project entry in the jjb/project-pipelines.yaml file in integration/config.

dev@laptop:config$ vim jjb/project-pipelines.yaml

# [...]
- project:
    name: my-project
    pipeline:
      - test
    jobs:
      # defines jobs:
      #   trigger-my-project-pipeline-test
      #   my-project-pipeline-test
      - 'trigger-{name}-pipeline-{pipeline}'
      - '{name}-pipeline-{pipeline}'
# [...]

There are two important lines here specific to your project—the rest is boilerplate. They are:

3. The name of your project.
5. The name(s) of the pipeline(s) you have defined in your .pipeline/config.yaml.

Tell Zuul to invoke your pipeline jobs to test project patchsets

Add new Zuul project test entries in the zuul/layout.yaml file in integration/config.

dev@laptop:config$ vim zuul/layout.yaml

Look under the projects: section in zuul/layout.yaml for your project, and add the trigger job under the test and gate-and-submit subsections.

# [...]
projects:
# [...]
  - name: my-project/repo/path/in/gerrit
    test:
      - trigger-my-project-pipeline-test
    gate-and-submit:
      - trigger-my-project-pipeline-test
# [...]

Commit your changes, and submit your patch for review!

dev@laptop:config$ git checkout -b add-my-project-test-pipeline
dev@laptop:config$ git add jjb/project-pipelines.yaml zuul/layout.yaml
dev@laptop:config$ git commit -em 'jjb: Define test pipeline for my-project' # and say a little more in the commit msg
dev@laptop:config$ git push origin HEAD:refs/for/master

(Optionally) Tell Zuul to invoke your publish pipeline

If your project includes a pipeline for publishing images to the registry and possibly even promoting that image in your Helm chart's values file, you probably want that to happen automatically to some extent. There are essentially two strategies for this:

1. Publish and/or promote upon every merged change (using Zuul's postmerge)
2. Publish and/or promote every time a new Git tag is pushed (using Zuul's publish)

For either strategy, you'll need to go back into zuul/layout.yaml and find your project's configuration section.

dev@laptop:config$ vim zuul/layout.yaml

Publish after every merged change

If you're taking the first approach, schedule your pipeline job to run during Zuul postmerge.

# [...]
projects:
# [...]
  - name: my-project/repo/path/in/gerrit
    test:
      - trigger-my-project-pipeline-test
    gate-and-submit:
      - trigger-my-project-pipeline-test
    postmerge:
      - trigger-my-project-pipeline-publish # assuming this is what you've named your publish pipeline
# [...]

Publish for every created/updated tag

If you'd prefer a more GitOps-y setup, schedule your pipeline job to run during Zuul publish.

# [...]
projects:
# [...]
  - name: my-project/repo/path/in/gerrit
    test:
      - trigger-my-project-pipeline-test
    gate-and-submit:
      - trigger-my-project-pipeline-test
    publish:
      - trigger-my-project-pipeline-publish
# [...]

Additionally, Pipelines that are invoked via Zuul's publish may make use of an additional variable exported by the setup stage called ${setup.tag} that holds the value of the newly created/updated Git tag.

pipelines:
# [...]
  publish:
    blubberfile: blubber.yaml
    stages:
      - name: production
        build: production
        publish:
          image:
            id: '${.imageID}'
            tags: ['${setup.tag}']