Migration from puppet 2.7.x to 3.y (where y should be 2 or higher) is not going to be pain-free, but we want it at least to be as safe as possible.

To this end, we have created a set of tools that could help us all in finding out the differences in produced catalogs between the two puppet versions.

At the moment the best way to check which differences there are between the catalogs produced by puppet 2.7 and puppet 3.x is to compile the modules with both versions and control the differences.

Things to watch out for

Here is an (under construction) list of things to watch out for while we migrate.

Dynamic scope is gone

The biggest change you will have to cope with are the changes in how puppet scope works, see scope and puppet for some details. Also see an internal discussion on this, with examples.

Dashes in class names are not allowed

Well, sort of. The trick is, you can include a class with dashes in its name, but you will not be able to reference ANY of its variables. With the dynamic scope lookup gone, this is going to make your classes useless. Don't use dashes in class names.

Templates should be written correctly

The second biggest change is in how variables are looked up and treated in templates; the changes are both in scoping and in syntax, see the puppetlabs template reference.

In short:

You should reference variables with @myvariable if that variable is defined in the same scope where the template is defined, or by using the scope.lookupvar function. See this blog post for a practical example.
Undefined values have changed behaviour;
If you are using functions within templates, your invokation should:
- be a child of the scope object
- use an array as argument even if you're passing on just one value.

Custom function argument must always be an array

This is painful, as if you have written a scalar function this will break your work completely. Say you had a function to capitalize a string

module Puppet::Parser::Functions
  newfunction(:myfunc, :type => :rvalue) do |args|
    fail 'myfunc() requires an argument' if args.empty?
    args.capitalize
  end
end

and you are using that in puppet as follows:

$lang="english"
$msg=capitalize($lang)

you need to transform it to a function working on arrays, in example:

module Puppet::Parser::Functions
  newfunction(:myfunc, :type => :rvalue) do |args|
    fail 'myfunc() requires an argument' if args.empty?
    fail 'myfunc() takes exactly one argument' if args.length > 1
    args.first.capitalize
  end
end

and correspondingly

$lang="english"
$msg=capitalize([$lang])

Puppet Catalogs compiler

compare-puppet-catalogs (inside operations/software repo) is a small utility to assist in the migration of your Puppet manifests from Puppet 2.7 to Puppet 3.0.

It has several levels of inceptions but should be easily runnable using the provided Vagrant file.

There is more information about catalog compilation in the puppet-compiler page.

Preparing the environment

First of all, provision and start the vagrant box

   vagrant up

Fetch a tar archive of our production facts, i.e. on palladium:

   sudo tar cJf /tmp/puppet-facts.tar.xz /var/lib/puppet/yaml

and copy it to the base directory of compare-puppet-catalog on your host.

Now, you should ssh into your vagrant box and become root

   vagrant ssh
   sudo -i
   export PATH="$PATH:/vagrant"
   # Set up your environment
   /vagrant/shell/helper install

This last command will take a while to execute. Now you can see the help of the command

   comparator -h

Comparator has three main working modes:

1) See the differences between catalogs compiled in puppet 2.7 and puppet 3 on the production branch:

   comparator --numthreads 1 \
       --nodes amssq55.esams.wikimedia.org,fenari.wikimedia.org

if you omit --nodes, it will run on a predetermined set of nodes (see below). If you have configured your vagrant box to have more than one core, raise the --numthreads parameter to speed up catalog compilation.

2) See the effects of a Gerrit changeset on the puppet catalogs by using the --change argument:

   comparator --numthreads 1 --change 130591 \
       --nodes amssq55.esams.wikimedia.org,fenari.wikimedia.org

3) Same as 1), but for a Gerrit changeset:

   comparator --numthreads 1 --change 130591 --transition\
       --nodes amssq55.esams.wikimedia.org,fenari.wikimedia.org

The run should produce, apart from some useless logs on stdout, a tree of results, that you will be able to browse from your browser at one of the following locations:

http://localhost:8080/html (case 1)

or

http://localhost:8080/change/<change_id>/html/ (case 2 or 3)

Default node selection

By default, comparator selects the nodes to run onto intelligently: it will inspect your site.pp (from the puppet tree to examine) and match all the nodes you have facts for to it; then it will pick only one (random) node matching any specific node definition. This way, we should have 99% coverage with tests running on less than 15% of the hosts.

License

GPLv2, see file LICENSE.