8.7 KiB
Title: K8s on NixOS - Chapter 2: Certificates (PKI) Date: 2025-03-14 Category: software Tags: certificates, ca, pki Slug: k8s-on-nixos-chapter2-pki Summary: In this chapter we will setup our CAs and create all the certificates we will need. Status: Draft
Things not covered in this guide
Certificate Revocation lists (CRL)
In a production setup you would probably build a CRL so that you can put your old or compromised certificates on it. That way all users of your PKI know which certificates are still valid and should be trusted. It is a good improvement for security. For the etcd cluster this could be configured like this:
services.etcd.extraConf = {
"CLIENT_CRL_FILE" = config.age.secrets.etcd-client-crl.path;
"PEER_CRL_FILE" = config.age.secrets.etcd-peer-crl.path;
};
Intermediate CAs
In a production scenario you would maybe create intermediate CAs. It can be a good idea to do that, so you can keep your root CA safe and secure in an offline location and use your intermediate for operational tasks. Maybe you have a bigger PKI infrastructure with more use cases. In the case of an etcd and k8s setup it makes little sense to do that. The security gain is minimal at best, so the operational overhead doesn't remedy the security benefits.
So if you want, you can create intermediate CAs, but I will skip this step. If you do, you have to adapt the
ca_constraints to have a max_path_len=1 for the root CAs and max_path_len=0 for the intermediate CAs. Make sure
to also set pathlenzero=true for the intermediate CAs.
Basic setup
We will have two Certificate Authorities. One for the Etcd cluster and another one for the Kubernetes cluster. That way a Kubernetes client won't be able to become an Etcd client by accident or the other way around.
Under the Etcd CA, will create a certificate for each Etcd peer and one for each Etcd client. Under the Kubernetes CA we will create certs for the API server, the controller manager, server and client certs for the kubelet, the kube-proxy and the scheduler. Most of them are client certs that will access the Kubernetes API server.
We will use cfssl to create our Certificate Authority. You could do the same with just openssl if
you prefer that. cfssl comes with baked in defaults that make it easier to get the security right and you can
configure them in easy to read JSON files. So if you're no cryptography expert, I would suggest using it. In the ./pki
directory of our repository we'll create a ca-config.json file like this:
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"root": {
"expiry": "87600h",
"usages": [
"signing",
"cert sign",
"crl sign",
"digital signature"
],
"ca_constraint": {
"is_ca": true,
"max_path_len": 0,
"max_path_len_zero": true
}
},
"client-server": {
"expiry": "8760h",
"usages": [
"digital signature",
"key encipherment",
"server auth",
"client auth"
]
},
"client": {
"expiry": "8760h",
"usages": [
"digital signature",
"key encipherment",
"client auth"
]
},
"server": {
"expiry": "8760h",
"usages": [
"digital signature",
"key encipherment",
"server auth"
]
}
}
}
}
It configures profiles for all types of certificates we're gonna need.
- Root certificate
- Client
- Server
- Client & Server
We will use eliptic curves for all of our certificates. I don't know if they're better then RSA. I just guess they are and I like that the file size is smaller. Ask a cryptography expert what you should use if you want a proper recommendation.
Create root Certificate Authorities (CA)
We will create two CAs. One for the etcd cluster and another one for the Kubernetes cluster. That way we avoid that some component of our Kubernetes cluster could try acting like an etcd node. The etcd cluster will only trust peers that are signed by the etcd CA.
Create a file ./pki/etcd-root/etcd-root-csr.json for the root CA of the etcd cluster:
{
"CN": "etcd-root-ca",
"key": {
"algo": "ecdsa",
"size": 521
},
"names": [
{
"C": "DE",
"L": "Berlin",
"O": "my-cluster",
"OU": "etcd",
"ST": "Berlin"
}
],
"ca": {
"expiry": "87600h",
"pathlen": 0,
"max_path_len_zero": true
}
}
Some of these settings are redundant. I'm not sure why it's needed but it only works this way. For our Kubernetes CA we
will create the file ./pki/k8s-root/k8s-root-csr.json with this content:
{
"CN": "k8s-root-ca",
"key": {
"algo": "ecdsa",
"size": 521
},
"names": [
{
"C": "DE",
"L": "Berlin",
"O": "my-cluster",
"OU": "k8s",
"ST": "Berlin"
}
],
"ca": {
"expiry": "87600h",
"pathlen": 0,
"max_path_len_zero": true
}
}
Create a client cert
As a last step we will create a client certificate for us. This cert will be used in kubectl to talk to the kubernetes
API server.
Here is the config file ./pki/k8s-admin/k8s-admin-csr.json for it:
{
"CN": "admin",
"key": {
"algo": "ecdsa",
"size": 521
},
"names": [
{
"C": "DE",
"L": "Berlin",
"O": "my-cluster",
"OU": "etcd",
"ST": "Berlin"
}
]
}
Automate all the things!
With our config files in place, let's create a Makefile that stores all the commands to generate certs, encrypt and
decrypt them properly. We'll put in in ./pki/Makefile:
.DEFAULT_GOAL := help
.PHONY: decrypt
decrypt: decrypt-etcd decrypt-k8s ## copy certificates and keys from agenix into pki dir
.PHONY: decrypt-etcd
decrypt-etcd:
cd ../secrets ; agenix -d etcd-root-crt.age > ../pki/etcd-root/etcd-root.pem
cd ../secrets ; agenix -d etcd-root-key.age > ../pki/etcd-root/etcd-root-key.pem
.PHONY: decrypt-k8s
decrypt-k8s:
cd ../secrets ; agenix -d k8s-root-crt.age > ../pki/k8s-root/k8s-root.pem
cd ../secrets ; agenix -d k8s-root-key.age > ../pki/k8s-root/k8s-root-key.pem
cd ../secrets ; agenix -d k8s-admin-crt.age > ../pki/k8s-admin/k8s-admin.pem
cd ../secrets ; agenix -d k8s-admin-key.age > ../pki/k8s-admin/k8s-admin-key.pem
.PHONY: encrypt
encrypt: encrypt-etcd encrypt-k8s ## copy certificates and keys from pki dir into agenix
.PHONY: encrypt-etcd
encrypt-etcd:
cd ../secrets ; cat ../pki/etcd-root/etcd-root.pem | agenix -e etcd-root-crt.age
cd ../secrets ; cat ../pki/etcd-root/etcd-root-key.pem | agenix -e etcd-root-key.age
.PHONY: encrypt-k8s
encrypt-k8s:
cd ../secrets ; cat ../pki/k8s-root/k8s-root.pem | agenix -e k8s-root-crt.age
cd ../secrets ; cat ../pki/k8s-root/k8s-root-key.pem | agenix -e k8s-root-key.age
cd ../secrets ; cat ../pki/k8s-admin/k8s-admin.pem | agenix -e k8s-admin-crt.age
cd ../secrets ; cat ../pki/k8s-admin/k8s-admin-key.pem | agenix -e k8s-admin-key.age
.PHONY: etcd_with_root
etcd_with_root: etcd-root ## Create all etcd certs incl. root CA
.PHONY: etcd-root
etcd-root: ## Create root CA for etcd
cfssl gencert -initca -config ca-config.json -profile root etcd-root/etcd-root-csr.json | cfssljson -bare etcd-root/etcd-root
openssl x509 -text -noout -in etcd-root/etcd-root.pem
.PHONY: k8s
k8s: k8s-admin ## Create all kubernetes certs w/o root CA
.PHONY: k8s_with_root
k8s_with_root: k8s-root k8s-admin ## Create all kubernetes certs incl. root CA
.PHONY: k8s-root
k8s-root: ## Create root CA for kubernetes
cfssl gencert -initca -config ca-config.json -profile root k8s-root/k8s-root-csr.json | cfssljson -bare k8s-root/k8s-root
openssl x509 -text -noout -in k8s-root/k8s-root.pem
.PHONY: k8s-admin
k8s-admin: ## Create client cert for kubernetes admin (That's you 🙋)
cfssl gencert -ca k8s-root/k8s-root.pem -ca-key k8s-root/k8s-root-key.pem -config ca-config.json -profile client k8s-admin/k8s-admin-csr.json | cfssljson -bare k8s-admin/k8s-admin
openssl x509 -text -noout -in k8s-admin/k8s-admin.pem
.PHONY: help
help: ## Display this help
@grep -h -E '^[a-zA-Z0-9_-]+:.*?## .*$$' $(MAKEFILE_LIST) | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
*[PKI]: Public Key Infrastructure *[CA]: Certificate Authority