PipelineLib/Guides/How to define a golang test pipeline
Welcome to How to define a Go test pipeline, a simple PipelineLib guide to get your Go project patchsets tested and linted.
Prerequisites
- A basic understanding of the Blubber image build tool. The Blubber tutorials can help you with that.
- An understanding of
.pipeline/config.yaml
basics, namely how to define a simple pipeline that will build a test image variant and run it. Some of the PipelineLib tutorials should help you learn these basics.
And of course, don't hesitate to get in touch with the WMF Release Engineering Team for help with following this guide.
Guide
The basic steps to getting your Go test/lint tasks running in CI using Blubber and PipelineLib are:
- Define image variants in
.pipeline/blubber.yaml
that invoke your test and lint entry points. - Define a new pipeline that includes stages for building and running your image variants.
- Tell our CI system about your new pipeline by following the How to configure CI for your project supplemental guide.
Define test and lint image variants
v4
of the Blubber configuration, it lacks Go specific directives. However, one can use the generic builder
variant configuration to efficiently build a Go image.dev@laptop:~$ cd src/project/foo
dev@laptop:foo$ mkdir -p .pipeline/
dev@laptop:foo$ vim .pipeline/blubber.yaml
version: v4
variants:
build:
base: docker-registry.wikimedia.org/golang:latest
apt: { packages: [ca-certificates, g++, gcc, git] }
builder:
requirements: [go.mod, go.sum]
command:
- sh
- -c
- >-
go mod download &&
go mod verify &&
go get -u golang.org/x/lint/golint
copies: [local]
runs:
insecurely: true
lint:
includes: [build]
entrypoint: [golint, ./...]
test:
includes: [build]
entrypoint: [go, test, ./...]
This .pipeline/blubber.yaml
gives you two runnable image variants, one that invokes golint ./...
and another that invokes go test ./...
. In addition, both variants use a base build
variant that installs commonly needed development packages from our APT repository and downloads/verifies your project's Go module dependencies.
build
variant in this case ensures that your runnable variants will share the same common image layers. Using the builder
directive to install Go modules ensures that layers for the project's dependencies are properly cached and rebuilt should only go.mod
or go.sum
be changed.Define a simple test pipeline
Now that you have two variants for linting and testing, define a test
pipeline that builds and runs them.
dev@laptop:foo$ vim .pipeline/config.yaml
pipelines:
test:
blubberfile: blubber.yaml
stages:
- name: lint
- name: test
- name: lint
is shorthand configuration to build and run a variant that matches the stage name. See the stage configuration reference for details.(Optionally) Perform lint and test in parallel
Depending on your testing requirements, you may wish to run these two (or more) tasks in parallel. Configuration of execution
is how you accomplish that.
dev@laptop:foo$ vim .pipeline/config.yaml
pipelines:
test:
blubberfile: blubber.yaml
stages:
- name: build
build: build
run: false
- name: lint
- name: test
execution:
- [build, lint]
- [build, test]
This configuration essentially means: There are three distinct stages in two branches of execution. Execute build
first, then branch out and execute lint
and test
independently in parallel. As a visual graph, it ends up looking something like:
[set-up] ↓ build ⇙ ⇘ lint test ⇘ ⇙ [tear-down]
execution
graph is defined by reading the pipeline configuration reference.Next up
Now that you have a test pipeline defined for your project, head on over to How to configure CI for your project to tell our CI how to run it for each patchset submitted to Gerrit.