Miscweb/Kubernetes migration steps

This page contains steps needed to migrate static miscweb sites from dedicated miscweb hosts to Kubernetes. This steps are for static sites only, dynamic applications (with databases for example) need additional steps.

Migration can be separated in some information gathering, migrating all files to GitLab, configure GitLab CI pipeline, deploying the service to Kubernetes and switching the user traffic to the new backend.

Information gathering

• Find stakeholders of service and discuss expected update frequency and method (self-deploying vs. task creation) as well as intention of migration
• Find all service names of service (some services use multiple names)
• Find source repository for static html files (searching for the service name and git::clone in puppet is a good starting point)

Migration to GitLab

• Import source repository to GitLab repos/sre/miscweb namespace by using "Import Repository by URL"
  • Use Gerrit anonymous HTTP URL as "Git repository URL", for example https://gerrit.wikimedia.org/r/wikimedia/annualreport
  • Use same project slug as before (but use lower-case letters!)
  • Hit "import project"

• Add the new GitLab project to the list of allowed projects

Setup GitLab CI pipeline

This step uses blubber, gitlab-ci and kokkuri to build and publish a container image.

• Add a .pipeline folder to your project and create a blubber.yaml file.
  • Make sure to add all of the static html and apache config to the container image. See annual report blubber file for example.
  • You can build and test the image locally using DOCKER_BUILDKIT=1 docker build --target annualreport -f .pipeline/blubber.yaml . and docker run -p 8080:8080 <image name>
• Add a .gitlab-ci.yml file which includes required kokkuri ci templates. See annual report .gitlab-ci,yaml file for example.
• Merge change to main branch to trigger a full build and release of the image:
  • Monitor pipeline under CI/CD > Pipeline in GitLab
• Your image should be available under https://docker-registry.wikimedia.org/repos/sre/miscweb/<project_slug>/tags/ after some time (update of registry frontend needs some minutes/hours).
• Pull your image and test it locally

Deploy to Kubernetes

Changes here are made to deployments-charts repo and on deployment server.

• Add the services as a new release to miscwebs helmfile.yaml
• Add the new release to every environment to miscwebs helmfile.yaml
• Add a values file for the new release (see values-annualreport.yaml)
  • make sure to use the correct main_app.image and main_app.version matching the newly created image
  • make sure to add the correct extraFQDNs to ingress.gatewayHosts
• Add the same FQDNs to admin_ng/values/main.yaml#93 miscweb.tlsExtraSANs
• Coordinate deployment of new admin config with ServiceOps
  • cd /srv/deployment-charts/helmfile.d/admin_ng/
  • kube_env admin <environment>
  • helmfile -e staging-eqiad -l name=namespace-certificates -i diff
  • helmfile -e staging-eqiad -l name=namespace-certificates -i apply
• Deploy new service to Kubernetes
  • cd /srv/deployment-charts/helmfile.d/services/miscweb/
  • kube_env miscweb <environment>
  • helmfile -e <environment> -i apply

Switch service traffic

The last step is to switch the actual user traffic.

• Change trafficserver backend mapping from to in trafficserver/backend.yaml hiera.
  • Use https://miscweb.discovery.wmnet:4111 for new Kubernetes backends
• Change existing monitoring to new discovery URL
  • move httpbb tests to miscweb-k8s.yaml
  • move Prometheus blackbox checks to profile::microsites::monitoring

Cleanup

• Remove service from dedicated miscweb hosts and profile::microsites puppet profile
• Add note to old Gerrit repository and archive project (so users use the new repo from now on)
• Adjust documentation