Machine Learning/LiftWing/API/Internal usage

This page explains how users internal to WMF production infrastructure can access Lift Wing API endpoints. External users (including Toolforge and Cloud VPS): see External_usage instead.

Types of internal endpoints

Once a model server is deployed on Lift Wing, it is available internally via two endpoints:

Production: https://inference.discovery.wmnet:30443/v1/models/{MODEL_NAME}:predict
Staging: https://inference-staging.svc.codfw.wmnet:30443/v1/models/{MODEL_NAME}:predict (staging)

Access internal endpoints

Use Machine_Learning/LiftWing#Current_Inference_Services to find the {MODEL_NAME} and {KUBERNETES_NAMESPACE}.

Set the HTTP host header for your caller/client to reflect the following format: {MODEL_NAME}.{KUBERNETES_NAMESPACE}.wikimedia.org

Query internal endpoints using the stat100x nodes to avoid access issues. See the following examples for additional details.

Examples

This section shows sample API calls in different languages, for a few different models.

Each model available through the Lift Wing API has unique input requirements and output formats. Refer to the model card and API reference for each specific model and endpoint to understand how to make requests and interpret the response.

Curl

The way to query enwiki-goodfaith model via curl:

aikochou@stat1004:~$ cat input.json 
{ "rev_id": 1083325118 }

aikochou@stat1004:~$ curl "https://inference.discovery.wmnet:30443/v1/models/enwiki-goodfaith:predict" -X POST -d @input.json -i -H "Host: enwiki-goodfaith.revscoring-editquality-goodfaith.wikimedia.org" -A "YOUR_APP_NAME (YOUR_EMAIL_OR_CONTACT_PAGE)" --http1.1
HTTP/1.1 200 OK
content-length: 209
content-type: application/json; charset=UTF-8
date: Mon, 31 Oct 2022 16:51:54 GMT
server: istio-envoy
x-envoy-upstream-service-time: 361

{"enwiki": {"models": {"goodfaith": {"version": "0.5.1"}}, "scores": {"1083325118": {"goodfaith": {"score": {"prediction": true, "probability": {"false": 0.033641298577500645, "true": 0.9663587014224994}}}}}}}

If you get error curl: (60) SSL certificate problem: unable to get local issuer certificate, using

--cacert /etc/ssl/certs/wmf-ca-certificates.crt to add the CA certificate.

If you get error curl: (56) Received HTTP code 403 from proxy after CONNECT from the stat100x nodes, try with

unset https_proxy to unset your http proxy environment variables.

Python

The way to query outlink-topic-model via python:

import os
import json
import requests

os.environ['REQUESTS_CA_BUNDLE'] = "/etc/ssl/certs/wmf-ca-certificates.crt"

inference_url = 'https://inference.discovery.wmnet:30443/v1/models/outlink-topic-model:predict'
headers = {
    'Host': 'outlink-topic-model.articletopic-outlink.wikimedia.org',
    'Content-Type': 'application/json',
    'User-Agent': 'YOUR_APP_NAME (YOUR_EMAIL_OR_CONTACT_PAGE)'
}
data = {"lang": "en", "page_title": "Wings of Fire (novel series)"}

response = requests.post(inference_url, headers=headers, data=json.dumps(data))

print(response.text)

If you get the following error message from a stat100x node:

requests.exceptions.ProxyError: HTTPSConnectionPool(host='inference.discovery.wmnet', port=30443): Max retries exceeded with url: /v1/models/outlink-topic-model:predict (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden')))

Try with unset https_proxy and run the script again.

JavaScript

The way to query enwiki-goodfaith model via JavaScript:

const liftWingInternalEndpoint = "https://inference.discovery.wmnet:30443/v1/models/enwiki-goodfaith:predict";
const appName = "YOUR_APP_NAME";
const email = "YOUR_EMAIL_OR_CONTACT_PAGE";
let headers = new Headers({
    "Host": "enwiki-goodfaith.revscoring-editquality-goodfaith.wikimedia.org",
    "Content-Type": "application/json",
    "Api-User-Agent": appName + " ( " + email + " )"
});
let data = { "rev_id": 1083325118 };

fetch(liftWingInternalEndpoint, {
    method: "POST",
    headers: headers,
    body: JSON.stringify(data)
})
.then(response => response.json())
.then(inferenceData => console.log(inferenceData));

Since this is an internal endpoint, it is best to query it using the stat100x nodes to avoid access issues.

Rate limits for internal usage

Reference documentation