This page contains information for common maintenance tasks we administrator do in Toolforge.
Is the Toolforge-specific version of the CloudVPS admin maintenance conterpart wikipage.
This section contains documentation on several Toolforge-specific admin scripts we have.
This script helps manage Grid Engine exec nodes by allowing for depool, repool, and check status, given an exec node hostname.
- exec-manage [status|depool|repool] exec_host_name
- exec-manage [list]
root@tools-sge-master-01:~# exec-manage status tools-sgeexec-1001.tools.eqiad.wmflabs [..]
This script is for the new Toolforge kubernetes cluster.
It creates a x509 cert for a given ServiceAccount that can be used to auth against the k8s API. It must be executed from a control plane node.
user@tools-k8s-control-3:~$ sudo wmcs-k8s-get-cert -help INFO: Usage of this script: /usr/local/sbin/wmcs-k8s-get-cert -h/--help - show help and exit /usr/local/sbin/wmcs-k8s-get-cert <svcname> - generate a x509 TLS cert from the kubernetes API /usr/local/sbin/wmcs-k8s-get-cert <svcname> -v - same, but in verbose mode
user@tools-k8s-control-3:~ $ sudo -i wmcs-k8s-get-cert myserviceaccount /tmp/tmp.JYFsVzDnX4/server-cert.pem /tmp/tmp.JYFsVzDnX4/server-key.pem
NOTE: x509 certs expire in 1 year usually. See also Portal:Toolforge/Admin/Kubernetes/Certificates
This is a script for the Toolforge Kubernetes cluster, deployed to the control plane nodes.
It will give a specified tool account a service account named $tool-obs that will allow a pod running as it to have "view" access to the entire cluster. This is for building tools similar to Openstack Browser for Kubernetes.
user@tools-k8s-control-3:~$ sudo -i # You'll want full root for k8s credentials root@tools-k8s-control-3:~# wmcs-enable-cluster-monitor <tool-name>
If that runs successfully, you'll want to run the tool as that service account using a script like k8s_webservice.sh instead of using webservice.
Important point about using a service account: it must use the credentials mounted at
/var/secrets/kubernetes.io/serviceaccount in the pod and not
$HOME/.kube/config or you'll get the tool permissions instead of the $tool-obs permissions. This is largely because of invoking the pod presets to get the $HOME variable set to /data/project/$tool and the NFS mounted.
This script is for the Toolforge kubernetes cluster.
It creates a k8s secret to hold x509 cert for a given ServiceAccount which you can use inside a pod to auth to the k8s API server.
user@tools-k8s-control-3:~$ sudo wmcs-k8s-secret-for-cert -h INFO: this script creates a k8s secret to hold a x509 cert for a given service account (pub/priv) which you can then use inside a pod, mounting the secret as a volume. Usage: /usr/local/sbin/wmcs-k8s-secret-for-cert [-n <namespace>] -s <secretname> -a <svcname> [-v] | -h -n namespace the namespace in which the secret will be created (uses default if not specified). -s secretname name of the secret being created. This is the name you use to mount it later on. -a svcname service name, doing RBAC auth for this service account. -v verbose mode. -h show help and exit.
user@tools-k8s-control-3:~ $ sudo -i wmcs-k8s-secret-for-cert -n kube-system -s mysecret -a myserviceaccount secret/mysecret created
This script is calling internally other admin script, wmcs-k8s-get-cert.
When the secret is created, it can be used in a pod like this:
apiVersion: apps/v1 kind: Deployment metadata: name: my-deployment namespace: my-namespace spec: template: metadata: name: my-whatever spec: serviceAccountName: my-serviceaccount volumes: - name: my-secret-cert secret: secretName: my-secret-cert containers: - name: my-pod image: whatever args: - --tls-cert-file=/etc/certs/cert.pem - --tls-private-key-file=/etc/certs/key.pem volumeMounts: - name: my-secret-cert mountPath: /etc/certs readOnly: true [..]
NOTE: X509 certs expire in 1 year. See also Portal:Toolforge/Admin/Kubernetes/Certificates
This script is used to chain all the required steps required to upgrade the farm of k8s worker nodes on a given kubeadm-based kubernetes cluster.
More info in Portal:Toolforge/Admin/Kubernetes/Upgrading_Kubernetes.
Other scripts you may find interesting.
This script is used to build and publish packages into aptly. More information in the packaging page.
You can locate it in the puppet tree: modules/toolforge/files/wmcs-package-build.py. It is intended to run from your laptop.