Portal:Toolforge/Admin/Envvars Service
Documentation of components and common admin procedures for Envvars Service.
Components
- Envvars client (source code): main entrypoint for users
- Envvars API (source code): main entry-point for clients (users use the cli)
- Envvars admission controller (source code): Injects the envvars into the new pods.
Alerts
None specific yet
- From the cloud UI: https://prometheus-alerts.wmcloud.org/?q=%40state%3Dactive&q=project%3D~^%28tools%7Ctoolsbeta%29
- From the prod UI: https://alerts.wikimedia.org/?q=team%3Dwmcs&q=project%3D~%28tools%7Ctoolsbeta%29
Dashboards
TBD
Main phabricator board
https://phabricator.wikimedia.org/project/board/539/
Administrative tasks
Starting a service
Envvars API
This lives in kubernetes, behind the API gateway. To start it you can try redeploying it. To do so follow Portal:Toolforge/Admin/Kubernetes/Components#Deploy (the component is toolforge-envvars-api).
You can monitor if it's coming up with the usual k8s commands:
root@toolsbeta-test-k8s-control-4:/srv/git/toolforge-deploy# kubectl get all -n envvars-api
NAME READY STATUS RESTARTS AGE
pod/envvars-api-564c68f5b4-mrxgg 2/2 Running 0 8d
pod/envvars-api-564c68f5b4-z8q99 2/2 Running 0 8d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envvars-api ClusterIP 10.101.128.128 <none> 8443/TCP 8d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envvars-api 2/2 2 2 8d
NAME DESIRED CURRENT READY AGE
replicaset.apps/envvars-api-564c68f5b4 2 2 2 8d
Envvars admission controller
Also a k8s component, follow Portal:Toolforge/Admin/Kubernetes/Components#Deploy (the component is envvars-admission-controller).
root@toolsbeta-test-k8s-control-4:/srv/git/toolforge-deploy# kubectl get all -n envvars-admission
NAME READY STATUS RESTARTS AGE
pod/envvars-admission-5d7698bbbd-n565j 1/1 Running 0 8d
pod/envvars-admission-5d7698bbbd-rdhgj 1/1 Running 0 8d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envvars-admission ClusterIP 10.106.169.195 <none> 443/TCP 8d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envvars-admission 2/2 2 2 8d
NAME DESIRED CURRENT READY AGE
replicaset.apps/envvars-admission-5d7698bbbd 2 2 2 8d
Stopping a service
Envvars API
Being a k8s deployment, the quickest way might be just to remove the deployment itself (will require redeploying to start again).
root@toolsbeta-test-k8s-control-4:~# kubectl get deployment -n envvars-api envvars-api -o yaml > backup.yaml # in case you want to restore later with kubectl apply -f backup.yaml
root@toolsbeta-test-k8s-control-4:~# kubectl delete deployment -n envvars-api envvars-api
For a full removal (CAREFUL! Only if you know what you are doing) you can use helm:
root@toolsbeta-test-k8s-control-4:~# helm uninstall -n envvars-api envvars-api
Envvars admission controller
Similar to envvars api:
root@toolsbeta-test-k8s-control-4:~# kubectl get deployment -n envvars-admission envvars-admission -o yaml > backup.yaml # in case you want to restore later with kubectl apply -f backup.yaml
root@toolsbeta-test-k8s-control-4:~# kubectl delete deployment -n envvars-admission envvars-admission
For a full removal (CAREFUL! Only if you know what you are doing) you can use helm:
root@toolsbeta-test-k8s-control-4:~# helm uninstall -n buildpack-admission buildpack-admission
Checking all components are alive
We don't have a unified dashboard yet, but for now you can check each component individually.
Envvars API
You can monitor if it's coming up with the usual k8s commands:
root@toolsbeta-test-k8s-control-4:/srv/git/toolforge-deploy# kubectl get all -n envvars-api
NAME READY STATUS RESTARTS AGE
pod/envvars-api-564c68f5b4-mrxgg 2/2 Running 0 8d
pod/envvars-api-564c68f5b4-z8q99 2/2 Running 0 8d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envvars-api ClusterIP 10.101.128.128 <none> 8443/TCP 8d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envvars-api 2/2 2 2 8d
NAME DESIRED CURRENT READY AGE
replicaset.apps/envvars-api-564c68f5b4 2 2 2 8d
Envvars admission controller
Also a k8s component, different namespace:
root@toolsbeta-test-k8s-control-4:/srv/git/toolforge-deploy# kubectl get all -n envvars-admission
NAME READY STATUS RESTARTS AGE
pod/envvars-admission-5d7698bbbd-n565j 1/1 Running 0 8d
pod/envvars-admission-5d7698bbbd-rdhgj 1/1 Running 0 8d
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/envvars-admission ClusterIP 10.106.169.195 <none> 443/TCP 8d
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/envvars-admission 2/2 2 2 8d
NAME DESIRED CURRENT READY AGE
replicaset.apps/envvars-admission-5d7698bbbd 2 2 2 8d