PKI/Cloud

From Wikitech
< PKI

At the time of writing there is no central cloud PKI service however there is a pki project which is intended to be used in development environments and tries to mimic the behaviour of the pki service in production. In order to make changes to this project you will need to either ask one of the project admins or be an admin yourself. Further, the project must run their own puppet master as we will be required use real secrets.

Adding a new project to pki-intermediate

The cloud hiera data already should already have reasonable defaults. however the class is disabled/absent by default as such you will at the vary least need to add the following into either the project or host hiera data (e.g. gerrit PS) when you actully want to start using the class 

profile::pki::client::ensure: present

However before that we first need to configure the auth key on the project puppet master and add authorise the project puppet agents use the API server. Please note that for various reasons, only projects with their own self-hosted puppet masters (like deployment-prep) can use the Cloud PKI infrastructure.

If that is your case, please follow the next two sections to configure your puppetmaster!

Adding the auth key to the project

All requests made to the API service are signed using a hmac key. Currently we use the same key across all projects and CA's however it is still relatively secret (at the time of writing only the pki and deployment-prep projects have access). As such we need to copy the secret key somewhere into the private hiera structure on the puppet master. By default there are two options for this either somewhere appropriate under /etc/puppet/private/hieradata or in /etc/puppet/secret/hieradata/{common,$labsproject}.yaml. The former is a git repo populated by the Labs private git repo as such you would need to add this as a local commit. DO NOT ADD TO labs/private. The later is just a file location on the puppet master which reduces some complexity. however note the directory likely doesn't exists by default. The following example uses the deployment-prep project. You will need to ask one of the pki admins for the $secret_value below 

$ sudo mkdir -p /etc/puppet/secret/hieradata
$ printf 'profile::pki::client::auth_key: %s\n' "$secret_value" | sudo tee /etc/puppet/secret/hieradata/common.yaml

Authorising puppet agents for a specific project

The API service alsoi authenticates connections using TLS client auth. To simplify things the pki::client tools are configured to use the puppet agent certificates to preform authentication. however to get this to work you need to add the localcacert for the project puppet CA and add it to Cloud Client Auth CA file. This file is used as the SSLCACertificateFile by Apache when validating client auth connections (e.g. gerrit change). you can retrieve the localcacert by running the following on one of the puppet agents (not the puppet master as they are often not agents of them self) 

 
$ cat $(sudo facter -p puppet_config.localcacert)

Using the pki service

Once your a projects agents are authorised to use they simply need to include profile::pki::client for there node or project and follow the standard guide for creating certificates. Also the intermediates in production should be avalible in the cloud environment

Creating new intermediate CA's

The guides on the main Root CA guide should be all that is required using the following servers, however for testig purposes consider using `WMF_test_intermediate_ca` we can add additional usages/profiles to this ca if needed

  • root CA: pki-root.pki.eqiad1.wikimedia.cloud
  • Intermediate/Multiroot CA: pki-intermediate.pki.eqiad1.wikimedia.cloud
  • secret store: pki-pm.pki.eqiad1.wikimedia.cloud:/var/lib/git/labs/private


Debugging

It is worth noting for debugging purposes that the pki endpoint TLS connections uses the puppet CA from the pki project as such connections to the the pki service need indicate that they trust this CA. The required CA.pem file is distributed via profile::pki::client and should be available at /etc/ssl/localcerts/pki_api_CA.pem. This means that if using the cfssl tools directly you need to specify -mutual-tls-client-cert and mutual-tls-client-key pointing to the puppet agent certs and -tls-remote-ca /etc/ssl/localcerts/pki_api_CA.pem e.g. 

 
$  sudo /usr/bin/cfssl gencert -config /etc/cfssl/client-cfssl.conf \
                               -tls-remote-ca /etc/ssl/localcerts/pki_api_CA.pem \
                               -mutual-tls-client-cert "$(sudo facter -p  puppet_config.hostcert)" \
                               -mutual-tls-client-key "$(sudo facter -p  puppet_config.hostprivkey)" \
                               -label WMF_test_intermediate_ca \
                               -profile server \ 
                               /etc/cfssl/csr/foobar-client_pki_eqiad1_wikimedia_cloud.csr

or with curl 

# Allow insecure server connections when using SSL
$ curl -k -d '{"label": "WMF_test_intermediate_ca"}' \
 --cert "$(sudo facter -p  puppet_config.hostcert)" \
 --key  "$(sudo facter -p  puppet_config.hostprivkey)" \
 https://pki-intermediate.pki.eqiad1.wikimedia.cloud:443/api/v1/cfssl/info
{"success":true,"result":{"certificate":"-----BEGIN CERTIFICATE-----\nMIIDvTCCAx+gAwIBAgIUG7m3jAm3mRg6slno3tD9a1NSC7UwCgYIKoZIzj0EAwQw\ngY0xCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1T\nYW4gRnJhbmNpc2NvMSIwIAYDVQQKExlXaWtpbWVkaWEgRm91bmRhdGlvbiwgSW5j\nMRcwFQYDVQQLEw5DbG91ZCBTZXJ2aWNlczEUMBIGA1UEAwwLV01GX1RFU1RfQ0Ew\nHhcNMjEwMzA1MTIxNjAwWhcNMjYwMzA0MTIxNjAwWjCBhTELMAkGA1UEBhMCVVMx\nFjAUBgNVBAcTDVNhbiBGcmFuY2lzY28xIjAgBgNVBAoTGVdpa2ltZWRpYSBGb3Vu\nZGF0aW9uLCBJbmMxFzAVBgNVBAsTDkNsb3VkIFNlcnZpY2VzMSEwHwYDVQQDDBhX\nTUZfdGVzdF9pbnRlcm1lZGlhdGVfY2EwgZswEAYHKoZIzj0CAQYFK4EEACMDgYYA\nBAF1tjSU8YdOg6F12DqapnqU5sY2B1A5U0DmvZYUs2uQD5kd2cKLKMaP1JssFue1\nn+byZyqJvbb/MaRymLn6YuBe7ADbVr9a+bggfCWvPrO8HeHYDSqDEQ42VPAP7Syl\nko5qC7xILAff/EIc+4djxlI4gFAx8qf/TkkdFu/eWBnM4Aq8xKOCAR4wggEaMA4G\nA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/AgEBMB0GA1UdDgQWBBQmTHIQ\nf33IlwKT6Owwv19LAor0qDAfBgNVHSMEGDAWgBRxHoSXgws9q2wI9x9rQn0SVs4Z\n3zBfBggrBgEFBQcBAQRTMFEwTwYIKwYBBQUHMAGGQ2h0dHA6Ly9wa2ktaW50ZXJt\nZWRpYXRlLnBraS5lcWlhZDEud2lraW1lZGlhLmNsb3VkL29jc3AvV01GX1RFU1Rf\nQ0EwUwYDVR0fBEwwSjBIoEagRIZCaHR0cDovL3BraS1pbnRlcm1lZGlhdGUucGtp\nLmVxaWFkMS53aWtpbWVkaWEuY2xvdWQvY3JsL1dNRl9URVNUX0NBMAoGCCqGSM49\nBAMEA4GLADCBhwJCAd4DMpHjs8B8ggZInwUrWBxEeOU9ZqaCHmGAqeRDcepB8xho\n2gVo7gqky0nckX+RH3aNgzJTcE8rb0wfIysLIOWHAkEdAJKqMIZ8gXumcRvvaWj9\nOar/KEfMXQOCslxgu5upHun/zPYJ5O/pqdkuvwt/o8tidC+uWHKytog9viltr901\nFA==\n-----END CERTIFICATE-----","usages":["signing","key encipherment","client auth"],"expiry":"96h"},"errors":[],"messages":[]}
# Verify peer
$ curl -d '{"label": "WMF_test_intermediate_ca", "profile": "server"}' \
 --cert "$(sudo facter -p  puppet_config.hostcert)" \
 --key  "$(sudo facter -p  puppet_config.hostprivkey)" \
 --cacert /etc/ssl/localcerts/pki_api_CA.pem \
 https://pki-intermediate.pki.eqiad1.wikimedia.cloud:443/api/v1/cfssl/info
{"success":true,"result":{"certificate":"-----BEGIN CERTIFICATE-----\nMIIDvTCCAx+gAwIBAgIUG7m3jAm3mRg6slno3tD9a1NSC7UwCgYIKoZIzj0EAwQw\ngY0xCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMRYwFAYDVQQHEw1T\nYW4gRnJhbmNpc2NvMSIwIAYDVQQKExlXaWtpbWVkaWEgRm91bmRhdGlvbiwgSW5j\nMRcwFQYDVQQLEw5DbG91ZCBTZXJ2aWNlczEUMBIGA1UEAwwLV01GX1RFU1RfQ0Ew\nHhcNMjEwMzA1MTIxNjAwWhcNMjYwMzA0MTIxNjAwWjCBhTELMAkGA1UEBhMCVVMx\nFjAUBgNVBAcTDVNhbiBGcmFuY2lzY28xIjAgBgNVBAoTGVdpa2ltZWRpYSBGb3Vu\nZGF0aW9uLCBJbmMxFzAVBgNVBAsTDkNsb3VkIFNlcnZpY2VzMSEwHwYDVQQDDBhX\nTUZfdGVzdF9pbnRlcm1lZGlhdGVfY2EwgZswEAYHKoZIzj0CAQYFK4EEACMDgYYA\nBAF1tjSU8YdOg6F12DqapnqU5sY2B1A5U0DmvZYUs2uQD5kd2cKLKMaP1JssFue1\nn+byZyqJvbb/MaRymLn6YuBe7ADbVr9a+bggfCWvPrO8HeHYDSqDEQ42VPAP7Syl\nko5qC7xILAff/EIc+4djxlI4gFAx8qf/TkkdFu/eWBnM4Aq8xKOCAR4wggEaMA4G\nA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/AgEBMB0GA1UdDgQWBBQmTHIQ\nf33IlwKT6Owwv19LAor0qDAfBgNVHSMEGDAWgBRxHoSXgws9q2wI9x9rQn0SVs4Z\n3zBfBggrBgEFBQcBAQRTMFEwTwYIKwYBBQUHMAGGQ2h0dHA6Ly9wa2ktaW50ZXJt\nZWRpYXRlLnBraS5lcWlhZDEud2lraW1lZGlhLmNsb3VkL29jc3AvV01GX1RFU1Rf\nQ0EwUwYDVR0fBEwwSjBIoEagRIZCaHR0cDovL3BraS1pbnRlcm1lZGlhdGUucGtp\nLmVxaWFkMS53aWtpbWVkaWEuY2xvdWQvY3JsL1dNRl9URVNUX0NBMAoGCCqGSM49\nBAMEA4GLADCBhwJCAd4DMpHjs8B8ggZInwUrWBxEeOU9ZqaCHmGAqeRDcepB8xho\n2gVo7gqky0nckX+RH3aNgzJTcE8rb0wfIysLIOWHAkEdAJKqMIZ8gXumcRvvaWj9\nOar/KEfMXQOCslxgu5upHun/zPYJ5O/pqdkuvwt/o8tidC+uWHKytog9viltr901\nFA==\n-----END CERTIFICATE-----","usages":["digital signature","key encipherment","server auth"],"expiry":"96h"},"errors":[],"messages":[]}

More details on using the API directly can be found on the cfssl docs page

Create your Own cloud PKI instance

This is not currently supported however the cloud instances are using the production profiles. specifically we have a pki-root server which runs profile::pki::root_ca and a pki-intermediate server running profile::pki::multirootca. All there hiera config (exuding secrets) can be found in the puppet production repo

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=PKI/Cloud&oldid=2169516"