Portal:Cloud VPS/Admin/Hiera

From Wikitech
Jump to navigation Jump to search

This page contains information on how hiera data is managed in Cloud VPS, which is a bit different from the main production hiera/puppet workflow.

Precedence of hiera keys

Hiera keys may come from different backends, not only puppet hieradata files.

You can set hiera data by creating/editing a wiki page on wikitech, by using Horizon or through diverse yaml-files in the operations/puppet and labs/private git repos. You can see the exact, always up-to-date resolution order at [1].

The following is a dump as of 2018-03-31:

  • Horizon (either at the instance level, prefixes, or project level)
  • Wikitech page Hiera:%{::labsproject}/host/%{::hostname} (deprecated in favor of Horizon)
  • Wikitech page Hiera:%{::labsproject} (deprecated in favor of Horizon)
  • operations/puppet git repo, path "hieradata/labs/%{::labsproject}/host/%{::hostname}.yaml"
  • operations/puppet git repo, path "hieradata/labs/%{::labsproject}/common.yaml"
  • operations/puppet git repo, path "hieradata/labs.yaml"
  • locally on puppetmasters at "/etc/puppet/secret/hieradata/%{::labsproject}.yaml"
  • labs/private git repo, path "hieradata/labs/%{::labsproject}/common.yaml"
  • labs/private git repo, path "hieradata/%{::labsproject}.yaml"
  • labs/private git repo, path "hieradata/labs.yaml"
  • operations/puppet git repo, path "hieradata/common.yaml"
  • locally on puppetmasters at "/etc/puppet/secret/hieradata/common.yaml"
  • labs/private git repo, path "hieradata/common.yaml"

Where %{::labsproject} is the project (e.g. 'tools') and %{::hostname} the instance name (e.g. 'tools-mail'). Note that providing Hiera settings per role is not possible on Cloud Services.

Only users with the projectadmin role can edit hiera data in Horizon and on Wikitech. Everyone can propose a patch in gerrit to change the hiera data in git, which can then be merged by all of SRE/Ops.

puppetmaster configuration

In order to select a backend/source for hiera data, puppetmasters have a /etc/pupppet/hiera.yaml configuration file with several backends in order, for example:

tools-puppetmaster-01:~# cat /etc/puppet/hiera.yaml
[...]
:backends:
  - httpyaml
  - mwyaml
  - nuyaml
[...]

In the example above:

  • nuyaml == local config files
  • mwyaml == wikitech
  • httpyaml == data that horizon writes on the puppetmaster

enc mechanism

The enc mechanism is a custom made API that serves up hiera config (it's what horizon talks to).

TODO: elaborate

Wikitech integration

Our intention is to kill the wikitech hiera keys definitions. But wikitech has editing history and Horizon doesn't (phab:T153036).

See also