User:BryanDavis/Kubernetes

Fun things I have learned how to do on the Toolforge Kubernetes cluster.

Redirect requests for tools.wmflabs.org/my-tool to my-tool.toolforge.org

This is obsolete. Use webservice --canonical instead of hand hacking.

Host based tool routing is live for Toolforge tools! What is not live yet is a a simple way to tell webservice that you want all traffic to your tool to be redirected to its new hostname as the canonical URL even when folks find old links. If you are using the 2020 Toolforge Kubernetes cluster, you can setup this redirection manually by changing the annotations section of the webservice generated my-tool-legacy Ingress object.

$ become my-tool
$ kubectl edit ingress my-tool-legacy
: editor opens with a lot of yaml in it. See below for the before/after versions

Annotations generated by webservice:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^(/my-tool)$ $1/ redirect;
    nginx.ingress.kubernetes.io/rewrite-target: /my-tool/$2

Annotations to redirect to my-tool.toolforge.org:

metadata:
  annotations:
    nginx.ingress.kubernetes.io/permanent-redirect: https://my-tool.toolforge.org/$2

The customization of the Ingress object will survive a webservice restart, but will need to be done again manually following a webservice stop; webservice --backend=kubernetes [type] start cycle.

Make a tool redirect to another tool WITHOUT running a webservice

This trick relies on configuring the nginx-ingress system to handle the redirection.

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{TOOL_NAME}}-redirect
  namespace: tool-{{TOOL_NAME}}
  labels:
    name: {{TOOL_NAME}}-redirect
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/permanent-redirect: https://{{TARGET_TOOL_NAME}}.toolforge.org/$1$is_args$args
spec:
  rules:
    - host: {{TOOL_NAME}}.toolforge.org
      http:
        paths:
          - pathType: Prefix
            path: /(.*)$
            backend:
              service:
                name: unused
                port:
                  number: 8000

$ kubectl apply --validate=true -f ingress.yaml
ingress.extensions/grid-jobs-redirect created

Run multiple pods for your webservice

This is obsolete. Use webservice --replicas instead of hand hacking.

Have a webservice that is under a lot of load? Scale it up!

$ kubectl scale --replicas=2 $(kubectl get deployment -o name)

Run a cronjob

Help:Toolforge/Jobs framework is almost certainly a better way to do this these days.

cronjobs.yaml

---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: archivebot
  labels:
    name: archivebot
    toolforge: tool
spec:
  schedule: "34 2 2 * *"  # 02:34 on the 2nd of each month
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 0
  failedJobsHistoryLimit: 1
  jobTemplate:
    spec:
      template:
        metadata:
          labels:
            toolforge: tool
        spec:
          containers:
          - name: archivebot
            workingDir: /data/project/bd808-pywikibot
            image: docker-registry.tools.wmflabs.org/toolforge-python37-sssd-base:latest
            args:
            - /usr/bin/python3
            - /data/project/shared/pywikipedia/core/scripts/archivebot.py
            - User:BryanDavis/archivebot
            env:
            - name: PYWIKIBOT_DIR
              value: /data/project/bd808-pywikibot
            - name: HOME
              value: /data/project/bd808-pywikibot
          restartPolicy: Never

$ /usr/bin/kubectl apply --validate=true -f cronjobs.yaml
cronjob.batch/archivebot configured

Attach to a running pod

webservice --backend=kubernetes [runtime] shell will give you a shell inside a fresh Kubernetes pod, but sometimes you really want to poke around inside the pod that is running your webservice or bot.

$ kubectl exec -it $(kubectl get pods --output=jsonpath={.items..metadata.name}) -- /bin/bash

Launch an interactive shell in the cluster

shell.sh

#!/bin/sh
exec kubectl run interactive \
  --image=docker-registry.tools.wmflabs.org/toolforge-python37-sssd-base:latest \
  --restart=Never \
  --command=true \
  --labels='toolforge=tool' \
  --rm=true \
  --stdin=true \
  --tty=true \
  -- "$@"

Tail logs for multiple pods/containers

$ kubectl logs --all-containers=true --since=10m -f deployment.apps/toolhub-main
Found 2 pods, using pod/toolhub-main-856566f7d8-nmxwb
...
$ kubectl logs -f --since=10m --all-containers=true --max-log-requests=10 -lapp=toolhub
...