PKI/Debugging

Create a local CA

The following documents how you can create a local ca and start creating certificates for testing with cfssl.

first of we need to create a CA for so we need to create a ca.json file describing the ca csr e.g.

{
  "CN": "example_ca",
  "hosts": [
    "example_ca"
  ],
  "key": {
    "algo": "ecdsa",
    "size": 521
  },
  "names": []
}

Then generate the CA pem files

$ cfssl gencert -initca ca.json | cfssljson -bare ca                                
2022/12/13 17:00:07 [INFO] generating a new CA key and certificate from CSR
2022/12/13 17:00:07 [INFO] generate received request
2022/12/13 17:00:07 [INFO] received CSR
2022/12/13 17:00:07 [INFO] generating key: ecdsa-521
2022/12/13 17:00:07 [INFO] encoded CSR
2022/12/13 17:00:07 [INFO] signed certificate with serial number 148316615946698894478228238454444079242258482333

This file creates the ca.pem and ca-key.pem files we will use to sign the certs below, but before that we need to create a basic config that will by default create a client mTLS certificate but also has a server profile for the server end

{
    "auth_keys": {},
    "signing": {
        "default": {
            "usages": [
                "signing",
                "key encipherment",
                "client auth"
            ],
            "expiry": "672h"
        },
        "profiles": {
            "server": {
                "expiry": "672h",
                "usages": [
                    "digital signature",
                    "key encipherment",
                    "server auth"
                ]
            }
        }
    }
}

We also create the server.json and client.json csr json files

{
  "CN": "client.example.org",
  "hosts": [
    "client.example.org"
  ],
  "key": {
    "algo": "ecdsa",
    "size": 521
  },
  "names": []
}

{
  "CN": "server.example.org",
  "hosts": [
    "server.example.org"
  ],
  "key": {
    "algo": "ecdsa",
    "size": 521
  },
  "names": []
}

and we uses theses files to create the necessary certificates

$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=config.json client.json | cfssljson -bare client
2022/12/13 17:03:58 [INFO] generate received request
2022/12/13 17:03:58 [INFO] received CSR
2022/12/13 17:03:58 [INFO] generating key: ecdsa-521
2022/12/13 17:03:58 [INFO] encoded CSR
2022/12/13 17:03:58 [INFO] signed certificate with serial number 289403862863531179502632004600427887401919831808
$ cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=config.json -profile=server server.json | cfssljson -bare server
2022/12/13 17:03:31 [INFO] generate received request
2022/12/13 17:03:31 [INFO] received CSR
2022/12/13 17:03:31 [INFO] generating key: ecdsa-521
2022/12/13 17:03:31 [INFO] encoded CSR
2022/12/13 17:03:31 [INFO] signed certificate with serial number 117612018245647437513655227146842214832319987239

Finally we can validate theses certificates with openssl x509 -in cert.pem -noout -text

Alt DNS names

At the time of writing the pki::multirootca use the puppet agent certificates to provide authentication. As the pki service listens on pki.discovery.wmnet we make use of the puppet dns_alt_names configuration. This can cause problems if rebuilding the server as this options is not currently supported by the reimage scripts. As such it is recommended to follow the following steps when (re)building

• first move the host into the spare::system role this allows you to use the reimage scripts to rebuild the host and prevents it getting stuck
• once up move the host into the pki::multirootca role and run puppet. you should see a change like the following (although puppet will fail)

--- /etc/puppet/puppet.conf.d/10-main.conf      2021-03-25 11:19:13.680926176 +0000
+++ /tmp/puppet-file20210330-30308-1y7a7nd      2021-03-30 11:46:43.552449866 +0000
@@ -14,7 +14,7 @@
 server = puppet
 
 ca_server = puppetmaster1001.eqiad.wmnet
-
+dns_alt_names = pki.discovery.wmnet
 daemonize = false
 http_connect_timeout = 60
 http_read_timeout = 960

• once this is in-place run the sre.puppet.renew-cert cookbook to regenerate the new cert

$ sudo  cookbook sre.puppet.renew-cert --allow-alt-names  pki1001.eqiad.wmnet

• finally run puppet on the pki servers