WIP

Testing model.py locally using Docker

In T301766, we want to support extended outputs to reach feature parity with ORES, like: https://ores.wikimedia.org/v3/scores/enwiki/186357639/goodfaith?features where "?features" adds the full features output in the response. In the case, we need to change code in model.py. The following steps show how we can run Docker image and test the code locally, instead of on the ml-sandbox. In this way, one could keep working on their favorite text editor (even the UI-based ones).

Let's say we want to implement the feature for editquality model.

First step is to pull the editquality docker image from wikimedia docker registry:

➜  ~ git pull docker-registry.wikimedia.org/wikimedia/machinelearning-liftwing-inference-services-editquality:stable

Next we download the model binaries from the Github repo. Suppose we download the model to /home/user/models, Also put the model.py to be tested in /home/user/dev/inference-test. In model.py, remember to change the path to load model:

def load(self):
    with open("/models/enwiki.goodfaith.gradient_boosting.model") as f:
        self.model = Model.load(f)

We will need to mount both in the container and publish container's port to the host:

➜  ~ docker run -p 8080:8080 -it -v "$(pwd)"/models:/models -v "$(pwd)"/dev/inference-test:/inference-code --entrypoint=/bin/bash docker-registry.wikimedia.org/wikimedia/machinelearning-liftwing-inference-services-editquality:stable
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
somebody@3619e76998b9:/srv/editquality$

Now you enter a container, you will see somebody@3619e76998b9:/srv/editquality$ in the prompt. For editquality, two environment variables are required, one is the INFERENCE_NAME (model name you want to test) and another one is WIKI_URL:

somebody@3619e76998b9:/srv/editquality$ export INFERENCE_NAME=enwiki-goodfaith
somebody@3619e76998b9:/srv/editquality$ export WIKI_URL=https://en.wikipedia.org

After setting env variables, we can run the model.py

somebody@3619e76998b9:/srv/editquality$ python3 /inference-code/model.py

You should see something like:

...
[I 220310 14:49:26 kfserver:150] Registering model: enwiki-goodfaith
[I 220310 14:49:26 kfserver:120] Setting asyncio max_workers as 8
[I 220310 14:49:26 kfserver:127] Listening on port 8080
[I 220310 14:49:26 kfserver:129] Will fork 1 workers

Open another terminal, query the model server via curl:

➜  ~ cat input.json
{"rev_id": 123456}

➜  ~ curl localhost:8080/v1/models/enwiki-goodfaith:predict -X POST -d @input.json
{"predictions": {"prediction": true, "probability": {"false": 0.03387957196040836, "true": 0.9661204280395916}}}%