Jump to content

Release Engineering/Runbook/Puppet patches

From Wikitech

This is a runbook for testing and staging Puppet patches on servers in the Beta Cluster.

Meta

Writing a patch

See Puppet coding.

Testing a patch

When submitting a patch for operations/puppet.git, Jenkins typically reports within a minute or two with the results of syntax, coding convention, and unit tests.

Staging a patch

Before we deploy a patch to production, there's two kinds of tests we apply:

  1. Puppet compiler tests. This asks Puppet to simulate what would happen given all the production realm variables. This identical to what would happen in actual production, if applied to a clean install of the HEAD-1 state on a fresh server and no private overrides.
  2. Beta Cluster testing. This will actually apply the patch to a real server in the Beta Cluster. Catches everything that would happen on a real server. But, it runs with the betacluster realm variables instead of production. So there may be intentional differences.

Puppet compiler

Prerequisites:

  • Wikimedia Developer account (same as Gerrit account), with "wmf" or "nda" user group.

Steps:

  • Use the build form for the puppet-compiler job on Jenkins.
  • Enter the Gerrit change number.
  • Enter the list of nodes to simulate before/after. For our patches this is usually: webperf1001.eqiad.wmnet,webperf1002.eqiad.wmnet,webperf2001.codfw.wmnet,webperf2002.codfw.wmnet
  • Start the build and view its console output. Once done, review its result. (example)
  • Share a link to the puppet compiler result in a comment on Gerrit.

Beta Cluster testing