Jump to content

User:TiagoLubiana/Uploading a Flask app on Toolforge

From Wikitech

This is a modification of Help:Toolforge/My first Flask OAuth tool for a Flask app that is already built and deployed elsewhere.

Overview

Python webservices are used by many existing tools. Python is a high-level, interpreted programming language with many available libraries for making webservices and integrating with MediaWiki.

This stub webservice is designed to get a sample Python application installed onto Toolforge as quickly as possible. The application is written using the Flask framework.

The guide will teach you how to:

  • Upload code to a the Toolforge server
  • Run a Python 3 WSGI webservice on Kubernetes
  • Have a persistent home for your Wikimedia-related Flask too

Getting started

Prerequisites

Skills

Accounts

Resources

  • A python Flask app running smoothly on your machine and hosted on GitHub (or any other online repository) that is related to Wikimedia
  • The correct tool layout.

Toolforge uses a WSGI configuration designed to make it easy to deploy a typical Python webservice. This configuration uses a 'convention over configuration' design with the following expectations:

  • The application will have a WSGI entry point in $HOME/www/python/src/app.py in a variable named app.
  • Python libraries will be loaded from a virtualenv located in $HOME/www/python/venv.
  • Logs will be written to $HOME/uwsgi.log (no need to create)
Expected file layout
$HOME
 └─ www
     └─ python
         └─ src
             ├─ app.py
             └─ requirements.txt

Step-by-step guide

  • Step 1: Create a new tool account
  • Step 2: Clone the application to your acount

Step 1: Create a new tool account

  1. Follow the Toolforge quickstart guide to create a Toolforge tool and SSH into Toolforge. In short:
    1. You can create a new tool at https://toolsadmin.wikimedia.org/tools/. Wait a bit for the changes to kick in.
    2. Run $ ssh <your-user>@login.toolforge.org
    3. Run $ become <TOOL NAME> to change to the tool user.

Step 2: Create a basic Flask WSGI webservice

Before uploading the tool, make sure it follows a standard kubernetes layout.

Start the webservice

Clone your application from GitHub (or your remote repository of choice):

$  export YOURTOOL="your_tool_name"
$  export YOURUSER="your_github_usermane"
$ git clone https://github.com/$YOURUSER/$YOURTOOL
$ cp -r $YOURTOOL/* .
$ rm -rf  $YOURTOOL
Set up the kubernetes webservice and set up a virtual environment

The virtual environment allows the tool to install Python libraries locally without needing a Toolforge administrator's help. The default webservice configuration will automatically load libraries from $HOME/www/python/venv.

Start the webservice
$ webservice --backend=kubernetes python3.11 start
Starting webservice.

The webservice will run on Kubernetes. A Kubernetes shell is required to create the virtual environment.

This will ensure that the version of Python the virtual environment uses matches the version of Python used by the Kubernetes runtime.

$ webservice --backend=kubernetes python3.11 shell
If you don't see a command prompt, try pressing enter.
$ python3 -m venv $HOME/www/python/venv
$ source $HOME/www/python/venv/bin/activate
$ pip install --upgrade pip
Add Flask to the virtual environment

Assuming you have your Python dependencies on a requirements.txt, proceed to add them to the virtual environment.

$ pip install -r $HOME/www/python/src/requirements.txt
Collecting flask (from -r www/python/src/requirements.txt (line 1))
[...]
Successfully installed [...]

The initial virtual environment is now set-up. Exit out of the Kubernetes shell and return to the SSH session on the bastion.

$ exit

Now restart the webservice

$ webservice restart
Restarting webservice...

Once the webservice has restarted, navigate to https://<TOOL NAME>.toolforge.org/ in your web browser to see the Flask app. And that is it!

Update your tool

Whenever your flask app has updates, make sure to push them to the git repository. Then, just access your toolforge account as specified above, and once more `git clone` your repository.

Then, you can synchronize your repositories, for example by running . 

$git clone https://github.com/$YOURUSER/$YOURTOOL
$ rsync -r $YOURTOOL .
$ rm -rf $YOURTOOL
$ webservice restart

It can also be run as a one line of code:

$export YOURUSER=YOURUSER && export YOURTOOL=YOURTOOL && git clone https://github.com/$YOURUSER/$YOURTOOL && rsync -r $YOURTOOL/* . && rm -rf $YOURTOOL && webservice restart

If the tool updates also include new dependencies, you should launch the kubernetes shell and follow the workflow described above.

See also

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=User:TiagoLubiana/Uploading_a_Flask_app_on_Toolforge&oldid=2211000"