Kubernetes/Deployment Charts

From Wikitech
Jump to navigation Jump to search

Deployment Charts

All of our existing Kubernetes clusters make use of a single git repository located at operations/deployment-charts:

This repository is organized as follows:

  • Helm charts in /charts
  • Helmfile deployments in /helmfile.d
  • Common templates in /_scaffold
  • Template modules in /modules
  • JSON schemas of all CustomResourceDefinition objects in /jsonschema
  • Any custom deployments that cannot use helm for any reason, in /custom_deploy.d
  • Supporting test files and scripts

See the README file in the repository for more detailed information.

Helm Charts

A Helm chart is a set of files that describes a packaging together of Kubernetes resources so that they can be easily deployed as a unit and updated programmatically.

Once committed to the master branch, our helm charts are automatically published to ChartMuseum.

Common Templates

We have developed a set of common templates for our helm charts.

Testing a Chart

Our CI will run a set of tests to validate a chart. In order to do that locally you will need:

  • Python 3.9 or higher
  • python3-yaml linux package or PyYAML (via pip)
  • Ruby 3.0 or higher
  • rake, a ruby make-like utility
  • helm3
  • docker or another container engine

To ensure the validity of our templates, we use two sets of fixtures.

  • fixtures under charts/mychart/.fixtures for various test cases
  • fixtures under .fixtures providing general data (eg mariadb sections, kafka_brokers etc) for misc enviroments (staging, eqiad, etc)
    • run rake refresh_fixtures to get the latest ones from our puppet repo.

Quick tests

To quickly test against a specific fixture:

helm template  --values .fixtures/general-eqiad.yaml \
     --debug  -f charts/mychart/.fixtures/tls_enabled.yaml \

CI tests

To Validate against our CI:

rake run_locally["check_charts[lint/validate\,mychart]"]