Data Platform/Systems/Kubernetes
< Data Platform | Systems
Placeholder
How to upgrade
Because we have persistent volumes, the dse-k8s cluster upgrade path is a bit different than the primary cluster at WMF (wikikube). Rather than destroying and replacing the clusters, we do rolling restart so we don't lose state.
Known issues
Certificate objects are not being recreated after the upgrade. Check each namespace (kubectl get cert). If it returns nothing, then you should do a `helm rollback` to the currently-deployed version:
brouberol@deploy1003:~$ k get certificate
No resources found in datahub namespace.
$ helm status datahub | grep -i revision
REVISION: 8
brouberol@deploy1003:/srv/deployment-charts/helmfile.d/dse-k8s-services/datahub$ helm rollback production 8
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /etc/kubernetes/datahub-deploy-dse-k8s-eqiad.config
Rollback was a success! Happy Helming!
brouberol@deploy1003:/srv/deployment-charts/helmfile.d/dse-k8s-services/datahub$ k get certificate
NAME READY SECRET AGE
datahub-frontend-production-tls-proxy-certs True datahub-frontend-production-tls-proxy-certs 9s
datahub-gms-production-tls-proxy-certs True datahub-gms-production-tls-proxy-certs 9s