Obsolete:Beta cluster node setup

This page may be outdated or contain incorrect details. Please update it if you can.

It is outdated. We now have:

hiera to setup puppet variables on a per project basis
saltmaster / puppetmaster autosigning

Antoine "hashar" Musso (talk) 16:32, 21 October 2015 (UTC)

This page describes the steps involved in adding a normal client node in a labs project (using mw:Beta Cluster / deployment-prep as an example), and make it work with puppet & salt.

Preliminaries: Puppet role setup

At Special:NovaPuppetGroup, check if there is already a puppet group for the project. For Beta Cluster, there is already the Beta Cluster group, otherwise create one. Then, add the classes you'd like to make available via the 'Add class' link on the group.

Node setup

At https://wikitech.wikimedia.org/wiki/Special:NovaInstance, create a new instance in your project. In Beta Cluster, name it deployment-<role>01 or some other numeric suffix. If you don't see the option to add a new instance, log out & back in (some bug). If you get an error during instance creation, ask in #wikimedia-cloud. It's likely that the quota needs to be increased.

It typically takes 10-15 minutes until you actually can log in. In the meantime, ssh login attempts will just return 'Permission denied'. This is likely some issue with mounting. If it doesn't fix itself within 30 minutes or so, try rebooting the instance.

Node variable setup

Select the roles you'd like to assign (for their setup, see #Puppet_role_setup). The role-specific form fields are some ldap hack. They are not in any way connected to hiera, which is actually configured per project at Hiera:Deployment-prep.

Select role puppet::self
Fill in
puppetmaster

deployment-puppetmaster.eqiad.wmflabs

Further down, fill out the following variables in the per-instance screen:

deployment_server_override: deployment-bastion.eqiad.wmflabs
salt_master_finger_override: dd:d8:68:70:8c:65:a3:af:46:5c:3f:4f:d4:be:6c:71 (pub server key returned by salt-key --finger-all on the salt master aka deployment-salt for Beta Cluster)
salt_master_override: deployment-salt.eqiad.wmflabs

Click save, possibly wait for some time (at least until you can log in).

On the puppet master (deployment-puppetmaster)

Accept the client cert with

sudo puppet cert sign i-0000080f.pmtpa.wmflabs # use your instance here

On the node

Remove the cached salt key & re-run puppet:

sudo rm /etc/salt/pki/minion/minion_master.pub
sudo puppet agent -tv # or -td for more verbosity

(From the salt master page.)

On the salt master (deployment-salt)

Accept the salt key with:

sudo salt-key --accept-all

Trebuchet submodule checkouts

Make sure manifests/role/deployment.pp has 'checkout_submodules' => true, for your repository. This will set variables (including a line in .git/config) to this effect on both the deploy server & the clients.

See the trebuchet troubleshooting section if this still doesn't work.

Troubleshooting

Puppet errors

Check /var/log/puppet.log. Run sudo puppet agent -td.

On the puppet master (deployment-puppetmaster), check the state of /var/lib/git/operations/puppet.