Istio

Kubernetes

• Clusters
• Docker Image building
• Deployments
• Deployment Charts
• kubectl
• kubectl/Cheat Sheet
• Service ports
• Education and learning resources
• Troubleshooting
• Resource requests and limits

Policies and administration

• Administration
• Add a new service
• Remove a service
• Creating a new cluster
• Adding or removing nodes
• Infrastructure upgrade policy
• OCI Image Versioning Policy
• Upstream Helm charts policy
• Packages

Infrastructure

• Calico
• cert-manager
• ChartMuseum
• Deployment pipeline
• Docker-registry
• Dragonfly
• Helm packaging
• Ingress
• Istio
• Logging
• Metrics
• Container Runtime Engine

Local development

• Minikube
• Local-charts

Related

• Kubernetes on Toolforge
• Mirror of the Kubernetes v1.16 docs
• Mediawiki On Kubernetes

[]

From the upstream website: Istio extends Kubernetes to establish a programmable, application-aware network using the powerful Envoy service proxy. Working with both Kubernetes and traditional workloads, Istio brings standard, universal traffic management, telemetry, and security to complex deployments.

Istio binary sources

In order to deploy Istio to our Kubernetes clusters, we use three main sources for the binaries needed:

• istioctl Debian package (repo https://gerrit.wikimedia.org/r/admin/repos/operations/debs/istioctl)
• istio-cni Debian package (repo https://gerrit.wikimedia.org/r/admin/repos/operations/debs/istio)
• The istio-build Docker image (repo https://gerrit.wikimedia.org/r/admin/repos/operations/docker-images/production-images, docker-registry https://docker-registry.wikimedia.org/istio/build/tags/)

Let's review each source to understand why we need it and how it is used (at a very high level).

Istioctl

The istioctl binary is a Go-based tool shipped by upstream that replaces the need for a dedicated Istio Helm chart. We use it to deploy CRDs and k8s settings, bypassing helmfile, but keeping the configuration yamls in the same repo (see https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/custom_deploy.d/istio/). We chose to follow what upstream recommends, since upgrades and general maintenance is easier with istioctl.

The tool is able to perform its job because, behind the scenes, it embeds helm3 and several charts that are maintained by upstream. Every version of istio ships with its own istioctl, so we needed a way to have multiple versions of istioctl on the deployment node (to support the fact that not all kubernetes nodes will run with the same Istio version at any given time). We created a Debian package that collects multiple releases of Istio from Github, and that packages istioctl binaries named istioctl-#version (for example, istioctl-195).

Istio build Docker image

This is where the Istio binaries are built (proxyv2 and pilot) to then be included in the Docker images that Istio needs to operate. We mimic what upstream does with few extra settings, since our policy is to avoid using untrusted repositories like Dockerhub directly. More information about the build process in the build Dockerfile, but it is basically calling some targets of a Makefile.

Istio-cni

In order to implement a service mesh, upstream provides a way to inject Envoy sidecars to kubernetes pod to intercept (transparently) traffic and apply routing / traffic policies over it (most useful ones are proxying and circuit breaking). The trick that Istio uses it to leverage special binaries implementing the CNI interface to inject iptables policies when the pod is being created (more info https://istio.io/latest/docs/setup/additional-setup/cni/).

The CNI binaries need to be shipped to the Kubernetes nodes, and we use a dedicated Debian package for it (we chose not to follow the install-cni Daemonset approach listed in the upstream page due to the security capabilities that need to be granted to the pods).