Welcome to How to define a Go test pipeline, a simple PipelineLib guide to get your Go project patchsets tested and linted.

Prerequisites

A basic understanding of the Blubber image build tool. The Blubber tutorials can help you with that.
An understanding of .pipeline/config.yaml basics, namely how to define a simple pipeline that will build a test image variant and run it. Some of the PipelineLib tutorials should help you learn these basics.

And of course, don't hesitate to get in touch with the WMF Release Engineering Team for help with following this guide.

Guide

The basic steps to getting your Go test/lint tasks running in CI using Blubber and PipelineLib are:

Define image variants in .pipeline/blubber.yaml that invoke your test and lint entry points.
Define a new pipeline that includes stages for building and running your image variants.
Tell our CI system about your new pipeline by following the How to configure CI for your project supplemental guide.

Define test and lint image variants

As of v4 of the Blubber configuration, it lacks Go specific directives. However, one can use the generic builder variant configuration to efficiently build a Go image.

dev@laptop:~$ cd src/project/foo

dev@laptop:foo$ mkdir -p .pipeline/
dev@laptop:foo$ vim .pipeline/blubber.yaml

version: v4

variants:
  build:
    base: docker-registry.wikimedia.org/golang:latest
    apt: { packages: [ca-certificates, g++, gcc, git] }
    builder:
      requirements: [go.mod, go.sum]
      command:
        - sh
        - -c
        - >-
          go mod download &&
          go mod verify &&
          go get -u golang.org/x/lint/golint
    copies: [local]
    runs:
      insecurely: true
  lint:
    includes: [build]
    entrypoint: [golint, ./...]
  test:
    includes: [build]
    entrypoint: [go, test, ./...]

This .pipeline/blubber.yaml gives you two runnable image variants, one that invokes golint ./... and another that invokes go test ./.... In addition, both variants use a base build variant that installs commonly needed development packages from our APT repository and downloads/verifies your project's Go module dependencies.

Having a common build variant in this case ensures that your runnable variants will share the same common image layers. Using the builder directive to install Go modules ensures that layers for the project's dependencies are properly cached and rebuilt should only go.mod or go.sum be changed.

Define a simple test pipeline

Now that you have two variants for linting and testing, define a test pipeline that builds and runs them.

dev@laptop:foo$ vim .pipeline/config.yaml

pipelines:
  test:
    blubberfile: blubber.yaml
    stages:
      - name: lint
      - name: test

Doing - name: lint is shorthand configuration to build and run a variant that matches the stage name. See the stage configuration reference for details.

(Optionally) Perform lint and test in parallel

Depending on your testing requirements, you may wish to run these two (or more) tasks in parallel. Configuration of execution is how you accomplish that.

dev@laptop:foo$ vim .pipeline/config.yaml

pipelines:
  test:
    blubberfile: blubber.yaml
    stages:
      - name: build
        build: build
        run: false
      - name: lint
      - name: test
    execution:
      - [build, lint]
      - [build, test]

This configuration essentially means: There are three distinct stages in two branches of execution. Execute build first, then branch out and execute lint and test independently in parallel. As a visual graph, it ends up looking something like:

     [set-up]
         ↓
       build
     ⇙       ⇘
lint           test
     ⇘       ⇙
    [tear-down]

You can learn more about how the execution graph is defined by reading the pipeline configuration reference.

Next up

Now that you have a test pipeline defined for your project, head on over to How to configure CI for your project to tell our CI how to run it for each patchset submitted to Gerrit.