Kubernetes/Kubernetes Workshop/Step 1

Step 1 Outcomes

Get to know our python based batch application
Learn how to install docker
Basic docker usage
Learn how to install a desktop based kubernetes called minikube
Run the batch application on kubernetes/minikube via a dockerhub account

At the end you should have basic familiarity with docker and minikube.

Step 1 - Run a batch application on k8s

In order to run a batch application on k8s we have to develop the application and package it into a docker image first.

Our demo application is written in Python 3. It accesses the main pages of the English, German and Greek Wikipedias, calculates a checksum over the content of the page and publishes the checksum to an IRC channel.

The application can be downloaded from github.com/wkandek/pywpchksumbot. The hardcoded IRC channel name is #demok8sws ^connect on Libera.chat.

The application functionality is not important for the workshop and feel free to explore a different application or reimplement it in another programming language, but this one has been tested and serves as a good stand-in for a “real” application.

Hands-on: download or code the application and run it locally.

The following can be done on a local machine, a VM or a server. It is simple and should work on Linux and MacOS. The underlying systems have included a laptop with 4 cores and 16 GB RAM running Ubuntu 20.04 LTS, a WMCS VM (medium size, g2.cores2.ram4.disk40, 2 CPU/4 GB) running Debian buster and a DigitalOcean server with 8 CPUs and 32 GB running Ubuntu 18.04 LTS. If you decide to run a VM locally, make sure cpu, memory and disk are adequate: 2 CPUs, 4 GB memory and 40 GB disk works well. For python3 on the Mac look at: https://docs.python-guide.org/starting/install3/osx/

Download pywpchksumbot.py from https://github.com/wkandek/pywpchksumbot
python3 pywpchksumbot.py
Check results on the IRC channel #demok8sws

Once the application runs locally let’s package it as a docker image and run it under docker.

Hands-on: run the application under docker

Install Docker

Debian: https://docs.docker.com/engine/install/debian/

apt-get remove docker docker.io containerd runc # might fail, but they are benign
apt-get update
apt-get install apt-transport-https ca-certificates curl gnupg-agent \ software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io
usermod -aG docker <your user> # logout/in your user after this

Ubuntu: https://docs.docker.com/engine/install/ubuntu/

apt-get remove docker docker.io containerd runc # might fail, but they are benign
apt-get update
apt-get install apt-transport-https ca-certificates curl gnupg-agent \ software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
apt-get update
apt-get install docker-ce docker-ce-cli containerd.io
usermod -aG docker <your user> # logout/in your user after this

Mac OS X - https://docs.docker.com/docker-for-mac/install/ - the installation for Mac OS X is Mac specific with a DMG file, plus Docker has an GUI console, but the functionality is the same as with Linux
Test Docker
- docker run hello-world - runs a simple test image that prints hello world

$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:49a1c8800c94df04e9658809b006fd8a686cab8028d33cfba2cc049724254202
Status: Downloaded newer image for hello-world:latest

What happens in the “background”
- A docker Image is downloaded - images are then available locally for the next run and then run inside of a local container.
  - docker image ls
  - docker container ls -a
  - docker container logs <id>

Create the image for pywkchksumbot
- Via a Dockerfile
  - The Dockerfile (a file named “Dockerfile”) describes how the Docker image is built. It is helpful to document the components needed for the application and makes the build process reproducible.
- Interactively
  - Docker images can be “developed” interactively, by installing the needed software manually and saving the image once the desired state is reached. For complex images it is probably the more common way of development, but it is near impossible to reproduce after the fact without documentation.
- Create a sample Dockerfile in the directory that contains the pywpchksumbot.py file

FROM python
ADD pywpchksumbot.py /
CMD ["/pywpchksumbot.py"]

The first line specifies the base image that we want to use, in our case we will use the image named “python”. Container images are downloaded from docker image registries and when no registry is specified the default registry is the Docker Hub at https://hub.docker.com. Take a look at that site and find the image named “python”. There are other image registries, but the docker hub is the most popular: 3.7 M images as of July 2020. Wikimedia has a registry which can be used by specifying FROM docker-registry.wikimedia.org:xxx. Its content can be browsed at: https://dockerregistry.toolforge.org/

docker image build . Builds the docker image using the instructions in the Dockerfile

Docker image build .
Sending build context to Docker daemon  7.475MB
Step 1/3 : FROM python
latest: Pulling from library/python
31dd5ebca5ef: Pull complete
3ed641c4ae98: Pull complete
bcd57146431e: Pull complete
ac34a4d7c330: Pull complete
3b0a7e6f20fb: Downloading [=======>    ] 137MB/192.2MB
75d93399dc4e: Download complete
28242c4c1c7e: Download complete
698b2a81f0a6: Download complete
e4cb0ad02465: Download complete

Docker images are composed of layers that are stacked on each other. There are 9 layers for the python image. The pywpchksumbot.py makes its own layer, so there are 10 layers in the last image. Layers are cacheable, so the next build will not have to download the entire stack again.

docker image build .
Sending build context to Docker daemon  7.475MB
Step 1/3 : FROM python
latest: Pulling from library/python
31dd5ebca5ef: Pull complete
3ed641c4ae98: Pull complete
bcd57146431e: Pull complete
ac34a4d7c330: Pull complete
3b0a7e6f20fb: Pull complete
75d93399dc4e: Pull complete
28242c4c1c7e: Pull complete
698b2a81f0a6: Pull complete
e4cb0ad02465: Pull complete
Digest: sha256:a9be1e55b2d309084dc00f5e06302062aebf1bf1b03ec3b2f6c97050f264edca
Status: Downloaded newer image for python:latest
---> 3189819ced3e
Step 2/3 : ADD pywpchksumbot.py /
---> 54b67f7c3c7c
Step 3/3 : CMD ["/pywpchksumbot.py"]
---> Running in f093d50ae4c1
Removing intermediate container f093d50ae4c1
---> e83413aa8db8
Successfully built e83413aa8db8

docker image inspect <image id from docker image> shows the layers in the output

Run the image
- docker run <image id from docker image ls>
- Check on #demok8sws for output
Tag the image for better identification
- docker image tag <id> pywpchksumbot
- docker image ls
- docker run pywpchksumbot

Hands-on: Run the application under Minikube

There are a number of ways of running a non-production k8s cluster on a development machine. Minikube is a simple k8s distribution that can be used to do so. Minikube runs k8s as a separate docker setup or inside a virtual machine depending on setup choices. Note that you need a minimum of 2 CPUs and 4 GB memory on the machine that run Minikube. If you run as a standard user the user will need sudo capabilities and use sudo where needed. Alternatively you can work as root, for example if you are working on a dedicated VM.

Install minikube - a single host k8s implementation https://minikube.sigs.k8s.io/docs/start/ has good installation instructions for all common OSs. For example on LInux:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
minikube start

Install kubectl - application used to interact with k8s - pronounced cube control, popular also cube ctl or cube cuttle or see for other alternates pronunciations. It is just a single binary and can be downloaded and installed easily. minikukube knows how to install it as well, so that is the most cross platform way:

minikube kubectl -- get po -A

or by hand on Linux:

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod 755 kubectl
sudo mv kubectl /usr/local/bin

Test kubectl - https://kubernetes.io/docs/reference/kubectl/cheatsheet/
- kubectl get nodes - nodes are the servers available to run applications on
  - Should list 1 node
- kubectl run helloworld --image=hello-world
  - kubectl logs helloworld
  - kubectl get pods
Run pywpchksumbot - minikube pulls images by default from docker hub:
- Create an account on docker hub with <username on dockerhub>
- Create a repo
- Push the local image
  - docker build . --tag=<username on dockerhub>/pywpchksumbot
    - Or docker tag <imageid> <username on dockerhub>/pywpchksumbot
  - docker login
  - docker push <username on dockerhub>/pywpchksumbot
- kubectl run checksum --image=<your account>/pywpchksumbot --restart=Never
  - This works: kubectl run checksum --image=wkandek/pywpchksumbot --restart=Never
- Check on the IRC to see if it ran
  - kubectl logs checksum gets the programs output.

Once done, minikube can be shutdown with minikube stop.