Portal:Cloud VPS/Admin/notes/Puppet Possibilities

From Wikitech

Talking Puppet, Hiera, Openstack and process

Meeting Agenda

Old universe

  • Nova in many places for (nearly) all things
  • intermingled repo management
  • Current install specific groupings (nova-api and nova-network, etc)
  • Per host values for labtest in hiera and regex madness
  • hiera lookups buried in: roles, modules, templates
  • Overlapping requirements for components handled via require_package() a lot
  • Passing dicts of huge numbers of unrelated params to get to a few
  • Duplicate params at various levels of the hiera tree
  • Convoluted component interdependence
  • Associated configuration lumped into large blocks in roles or other
  • ferm rules that make it difficult to reason on cleanup as it's unclear why it existed
  • Inconsistent strategy on hostname => ip esp for ferm
  • data structured mingled in with code (labsdb, etc)
  • The same value would be named multiple things throughout a config tree

General

  • Three failover scenarios
  • labcontrol (host and cold) via hiera
  • controlled via hiera params setting a boolean in a profile and passed to modules
    class openstack2::keystone::service(
        $active,
    
  • labservices (partial host and cold)
  • labnet (host and cold) role swapping <== looking to deprecate


  • Deployment separations (deployments to come)
  • deployments may end up being regions?
  • deployments are made up of components
  • components can be upstream openstack specific or not
  • Why not use inheritance:
  • We are working on a holistics openstack / VPS product profile model
  • consistent naming across components glance/service.pp, keystone/service.pp
  • If it is parameterized per deployment is worth doing base and deployment profiles potentially under profiles/manifests/openstack
  • Understanding our Hiera use cases:
  • Labs hiera tree is (also) a nightmare.
  • expand_paths vs not
  • labs/private is an overloaded dual purpose shim
  • back end split
  • Labs hiera is a nightmare
  • Per deployment labs/private or equiv may be needed
  • Puppet norms for new style are not always set up for sucess
  • base_firewall, standard, remaining implicit override, hiera adding functionality, etc

Upstream to us Style


To Be Done

  • Issue of cross site deployment variables (db1047)
  • Issue of special failover with labnet
  • firewall rules buried in site.pp
  • tools stub zone on labservices persistence
  • Issue of cleanup for hiera
  • openstack2 => openstack
  • test failover for pairs

Moving forward

  • resolve() within ferm
  • per ferm update resolution instead of per puppet run
  • human readable in diffs
  • Our own sub-style consistency that is an extension but not an override
  • File and resource consistence etc
    • owner,group,mode,source,requires
  • manifest flow: include, package, file(s), service
  • This results in per deployment per component profiles and per deployment roles
  • Much easier to test in labtest without effecting main, and we SHOULD/need to/must
  • staging models mean more of an opportunity for productive meaningful refactor
  • testplan designations
  • "foo: bar for x". "testplan: baz?"
  • modules/roles/manifests/wmcs
  • potentially: modules/roles/manifests/wmcs/tenants
  • Anything
  • Functional approach to hiera and parameters. Ironic? :)
  • In the mid-term we must consolidate on WMCS managed hiera strategy
  • A large and intimidating Toolforge cleanup must happen
  • We will have to solve the puppet multi-tenancy problem in the next 2 years or die trying