Talk:Puppet/Hiera

How to associate node and role

In most cases, we want a set of servers to have very similar configuration, which is in most cases a 1:1 correspondence with the role class that is included in the node itself; but this is not always the case. So the solution is to have some "label" applied to the server, and then use it in looking up variables in hiera.

How we do this today

Right now, you would probably want to do it like this:

site.pp

node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
    include role::foo
}

hieradata/regex.yaml

foocluster:
  __regex: !ruby/regex /^mc10[0-9][0-9]\.eqiad\.wmnet$/
  mainrole: foo_runner

hieradata/mainrole/foo_runner.yaml

foo::queue_len: 100
foo::threads: 8
admin::groups:
  - "oompah_loompahs"
  - "apache"
...

Which, while pretty organized, has the non-secondary disadvantage of asking us to replicate regexes twice.

The simplest alternative

The simplest and most obvious alternative would be to have a node-scope variable to use in every stanza to define the role we're applying:

site.pp

node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
    $mainrole = "foo_runner"
    include role::foo
}

hieradata/mainrole/foo_runner.yaml

foo::queue_len: 100
foo::threads: 8
admin::groups:
  - "oompah_loompahs"
  - "apache"
...

The drawback of this solution is that anything evaluated before the node scope is entered will not be correctly affected by this. Also, it makes sites.pp ugly (but this is a personal taste of mine), and still forces us to declare a relationship that should be more relative to the applied role than to the nodes. Subgroup-specific settings could still be defined in one of the two preceding ways.

Another idea

We can make the role/node relationship unique; this will be ok in 99.9% of the cases, as long as it can be manually overridden. The main advantage of this is that we tie functionality to the role class, and not to the node group. We can also declare multiple roles, and we'll have a dedicated hiera backend for this

So we would have:

manifests/role/foo.pp

class role::foo {

    # All the code that usually is in the role class.
}

manifests/site.pp

node /^mc10[0-9][0-9]\.eqiad\.wmnet$/ {
    role foo
}

hieradata/role/foo.yaml

foo::queue_len: 100
foo::threads: 8
admin::groups:
  - "oompah_loompahs"
  - "apache"
...

which could be overridden in hieradata/role/eqiad/foo.yaml.

This has some slight caveats I can think of:

• If more than one role is defined in a node def, all the hiera lookups on the roles must be identical, or an error will be thrown
• Things that depend on this declared in the top scope (and not in the node scope) will not be able to access this.

hiera.yaml no longer used?

The introductory section mentions the following:

Every variable that puppet looks up in hiera will be searched via one or more backends (at the moment, two a yaml-based ones) according to what we configured in the hiera.yaml file in the base puppet directory.

Looking at the repo seems to suggest that hiera.yaml is not used but instead hieradata/ is used instead. Does this make more sense?

Every variable that puppet looks up in hiera will be searched via one or more backends (at the moment, two a yaml-based ones) according to what we configured in the hieradata/ directory (Located in the base directory of the Puppet repository).

I'm a newbie to Puppet so forgive me if I'm overlooking something. But given that this page was written in 2014 it seems more likely that something has changed here :) --BCornwall (talk) 23:07, 11 May 2022 (UTC)

I've also been confused by this. In particular, "two a yaml-based ones" looks like a there's some words missing, and I'm not sure what it's supposed to say. RoySmith (talk) 23:15, 26 May 2022 (UTC)

"The hiera.yaml file" refers to the per-environment .hiera.yaml files in https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/puppetmaster/files/. As far as I can tell the listing in "The lookup hierarchy" section is completely out of date. Majavah (talk!) 18:51, 2 June 2022 (UTC)

i reverted the changes to the yaml blocks in the organisation section. Theses blocks have been created to show a simplified view of the hiera hierarcy. In my experience most people find it quite hard to interpret the hiera.yaml file syntax as such i have tried to show more concisely what is actually happening. users can always go to git to get the accurate picture.

in relation to

>"The lookup hierarchy" section is completely out of date

Can you expand i made a few small edits today but it seems fairly accurate to me. that said I'm inclined to drop it in favour of the practical example section? Jbond (talk) 08:50, 3 June 2022 (UTC)

Proposal to migrate from Puppet Hiera to Puppet/Hiera

I've consolidated nearly all the Puppet-related documentation under a single Puppet parent, and I believe Hiera's documentation also belongs there. @Jbond:, since you've been the most active in this article, would you agree/disagree? --BCornwall (talk) 18:57, 29 June 2022 (UTC)

+1 sgtm thanks Jbond (talk) 08:53, 11 July 2022 (UTC)

Agreed that's a good idea. Done. Majavah (talk!) 11:16, 23 December 2022 (UTC)