Portal:Toolforge/Admin/Kubernetes/lima-kilo

From Wikitech

This page contains information on Toolforge lima-kilo, a development and testing environment for Toolforge that reproduces most of the system components used in production.

The lima-kilo project is based on ansible and lima, light-weight linux VMs compatible with both linux and macOS (Intel and ARM) hosts.

Repository: https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo

Rationale

Tracked in Phabricator
Task T326789

Developing some Toolforge kubernetes components can be challenging because Toolforge is not just a bare k8s deployment. It is Toolforge's kubernetes. There are custom admission webhooks, custom namespaces, a particular security model, some host files are expected to be present, etc.

The lima-kilo recreates most of this service context in a local environment in a way that, thanks to ansible and lima, is lightweight, robust, deterministic, idempotent and easy to use.

Lima-kilo uses kind for its kubernetes component. The lima guest VM runs Debian. So far, lima-kilo has been tested on various linux distributions and both Intel and ARM macOS. It should probably work on WSL2, but this is so far untested.

If you run into compatibility issues while trying to set up lima-kilo on your system, don't hesitate to ask for help.

The project was mildly inspired by MediaWiki-Vagrant.

Basic usage workflow

This section needs updating since we officially moved to lima VMs rather than installing lima-kilo directly on the host system. The most up-to-date setup instructions can be found in the project repository

Usage should be rather simple:

  1. Git clone the repository
  2. make sure there is a playbook for your desired environment combo, check the playbooks/ directory. If not, ask for help with implementing a new one.
  3. run the playbook, like ansible-playbook -KD playbooks/kind-install.yaml
  4. enjoy!

You may want to set up some local configuration to modify the lima-kilo behavior. This can be done in the ~/.toolforge-lima-kilo/userconfig.yaml file, see the README file for some hints.

Code conventions

Some informal code conventions:

  • Everything lima-kilo does, it should be capable of undoing.
  • Therefore, there should be install and uninstall playbooks for each environment.
  • Playbooks should be idempotent (ie, if running them twice, the second time is basically a NOOP). This is a core principle of ansible, but worth mentioning.
  • If an action is too invasive in the local system, a config option should be introduced to prevent/relax the action. Like installing / uninstalling packages that might be managed by hand by the user.
  • When introducing changes, be mindful of previous lima-kilo deployments others developers may have in their system. This is important for example when renaming resources.
  • Try to write playbooks as much OS-agnostic as possible. So the same playbook can be used in both OSX and Debian.

See also

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Portal:Toolforge/Admin/Kubernetes/lima-kilo&oldid=2165366"