This page summarizes the learnings of T414453

Label and MWApi Service Migration

With the planned replacement of Blazegraph as the WDQS triple index, the question arises of what to do with the custom extensions. WDQS provides a set of non-standard SPARQL extensions that may or may not be easily ported to another backend. The question is whether to continue such extensions into the future.

Here we look at the two most heavily used extensions (the label and mwapi SERVICEs), and how they could be realized with QLever or Virtuoso as a backend.

Label Service

The WDQS provides the wikibase:label service as a non-standard extension to SPARQL. It allows users to easily request labels, alt labels and descriptions for items in the query by simply setting language preferences and mentioning the required literals in the select clause.

A simple example can be seen below (for more details see the WDQS documentation).

The current implementation is realized as a Blazegraph service extension, and as such, is not portable to other triple-stores.

Qlever

As stated in QLever issue 1829, the system can handle rdfs:label queries efficiently. As such, the only reason to support the original wikibase:label service is convenience for existing users and support for existing tools/bots like listeria.

Extending the QLever SPARQL parser to include support for custom SERVICEs is quite cumbersome and will result in a high maintenance overhead in the future (See also the corresponding Github discussion). A patch to the vanilla QLever codebase would be required. This patch will most likely become outdated regularly, requiring potentially complex rewrites and fixes whenever Qlever internals change, even slightly.

Virtuoso

Virtuoso provides a variety of extension possibilities. The Virtuoso Sponger can be extended to fetch additional data from the Mediawiki API for example. Alternatively, custom server functions can be implemented and called directly from SPARQL. These extensions do not require patches to Virtuoso itself but can be added through Virtuoso’s own extension functionalities (be it Virtuoso QL or any other supported language).

As of 27 March 2026, code commits to the Virtuoso GitHub repository begin to provide support for both the mwapi and label services.

While this is a viable approach, taking it means a hard vendor-lock-in. The extensions are non-standard SPARQL and therefore are Virtuoso-specific.

A Generic Solution

A simple query rewrite proxy could translate wikibase:label service usage into proper SPARQL statements using filters on the language. This solution is independent of any choice of backend, so long as it supports SPARQL 1.1.

Example

SELECT ?sLabel
  WHERE {
    ?s wdt:P22 ?o .
    SERVICE wikibase:label {
      bd:serviceParam wikibase:language "fr,en" .
    }
  }

Could be rewritten as

SELECT ?sLabel
 WHERE {
   ?s wdt:P22 ?o .
   OPTIONAL {
     ?s rdfs:label ?sLabel .
     FILTER(LANG(?sLabel) = “fr”) .
   }
   OPTIONAL {
     ?s rdfs:label ?sLabel .
     FILTER(LANG(?sLabel) = “en”) .
   }
 }

A java-based proxy has been created as a POC that rewrites queries before sending them to the new triple-store backend.

Note that the rewritten query is quite specific in its instructions, is portable to other environments (supporting SPARQL 1.1) and would produce more complete query plans for analysis.

Preliminary Performance Tests

Some initial and very basic performance tests have been conducted with a set of 8 SPARQL queries (taken from the Wikidata query examples). All queries have been rewritten via the proxy and then executed on Qlever and Virtuoso instances containing both the main and scholary graphs. As a comparison all original queries were also run against the current WDQS live service (main only). Each set of queries has been run twice: an initial run and a secondary run directly after that which allowed the backends to make use of caching.

The results can be seen below.

Query	WDQS Blazegraph cold (ms)	WDQS Blazegraph wam (ms)	Qlever cold (ms)	Qlever warm (ms)	Virtuoso cold (ms)	Virtuoso warm (ms)
query1.sparql	616	187	141	123	3403	110
query2.sparql	30181	1598	16972	2182	15718 (query required small changes for it to work)	15570 (query required small changes for it to work)
query3.sparql	877	140	848	163	2803	66
query4.sparql	1378	1228	1849	248	8046	321
query5.sparql	887	186	1376	175	7991	237
query6.sparql	876	192	1065	196	842	222
query7.sparql	timeout	timeout	1170	492	6694	171
query8.sparql	1847	153	497	425	6474	27

These numbers are not intended as final performance tests, they merely act as a sort of smoke test to see the validity of the approach. Additional testing with real-live query data is planned (T418595).

MWApi Service

The MwApi service is another non-standard SPARQL extension, realized as a custom Blazegraph service/plugin. It allows sending requests to the Mediawiki action API via a SPARQL query.

Qlever

The MWApi service can not easily be replaced with standard SPARQL features. As such a simple query rewrite proxy (as proposed for the label service) cannot handle it.

However, as with a custom label service, patch maintenance would most likely be more than cumbersome.

Virtuoso

For Virtuoso the exact same argument as for the label service applies for the mwapi service. While Virtuoso is very flexible in terms of extensions, they mean non-standard SPARQL syntax and a hard vendor lock-in.

A Generic Solution

One possible solution would be a companion SPARQL protocol service which is called by QLever via the standard SPARQL SERVICE federation support. This service would simply perform the necessary MWApi calls and return the result via the SPARQL protocol to QLever (or any triple store that supports federation).

A simple query rewrite proxy could, however, be used to replace instances of

SERVICE wikidata:mwapi

with something like

SERVICE <https://wdqs-mwapi.wikimedia.org/>

for it to transparently work in QLever itself.

Example

Given WDQS query

SELECT DISTINCT ?name WHERE {
  hint:Query hint:optimizer "None" .
  VALUES ?search_string { 
      "\"Robyn Shirley Dixon\"" "\"Robyn Dixon\"" "\"Robyn S. Dixon\"" }
  BIND(LCASE(REPLACE(?search_string, '"', )) AS ?lc_string)
  SERVICE wikibase:mwapi {
    bd:serviceParam wikibase:api "Search";
                    wikibase:endpoint "www.wikidata.org";
                    mwapi:srsearch ?search_string .
    ?page_title wikibase:apiOutput mwapi:title .
  }
  BIND(IRI(CONCAT(STR(wd:), ?page_title)) AS ?item)
  ?item wdt:P2093 ?name .
  FILTER CONTAINS(LCASE(?name), ?lc_string) .
}

Qlever/Virtuoso would forward the following query to the new service:

SELECT ?search_string ?page_title {
  VALUES (?search_string) { 
    ("\"Robyn Shirley Dixon\"") ("\"Robyn Dixon\"") ("\"Robyn S. Dixon\"") }   
  bd:serviceParam wikibase:api "Search";
                  wikibase:endpoint "www.wikidata.org";
                  mwapi:srsearch ?search_string .
  ?page_title wikibase:apiOutput mwapi:title .
}

A query like this can be translated into Mediawiki action API requests. Alternatively any other Mediawiki API could be used, for example GraphQL.

Work on a POC (based on the original mwapi Blazegraph service extension) is on-going, including performance tests to verify the scalability and performance of the approach (T418600).

Another Generic Solution

Another solution would be to add the required data directly into the WDQS index, thus avoiding external look-ups.

Given the relatively small number of users that actually use the mwapi service (as compared to the label service), this solution sounds undesirable. It would mean a considerable increase of triples in the index, a potential hit to performance and scalability, while only benefiting a small number of users.

Data analysis

Bug:T417980

We performed exploratory analysis of the query logs to better understand how the wikibase:label and mwapi services are being used across the main and scholarly graphs. Here we report findings from query logs collected from January 1st to Feb 27, 2026.

For the main graph, 67% of queries invoke the label service, while 23% call mwapi. Usage of mwapi is highly concentrated: just 20 actors account for 98.67% of all queries using the service, and aggregated searx bots alone are responsible for 95.5% of those calls. Only 70 of the total queries in February use mwapi:action, the rest uses a fallback to other apis (search).

In contrast, wikibase:label shows a longer tail of adoption. Although searx is still the single largest actor by overall query volume and accounts for 32% of label calls, usage is more broadly distributed across clients compared to mwapi.

The scholarly graph presents a different picture, which aligns with expectations given its audience. There are no searx user agents in this dataset. Here, 24% of queries use the label service, while calls to mwapi are virtually nonexistent (<0.01%), with only 32 total queries observed. Label usage is again concentrated, with the top three actors (including federated queries originating from the main graph) accounting for 99.9% of label service calls.

As regards other services, the table below reports query distribution across Graph and custom services. Bd:sample, bd:slice and gas:service are included as a follow up to User:AWesterinen/Blazegraph Features and Capabilities - Wikitech

Graph	scholarly_articles	wikidata_main
Queries (total)	6,040,661	766,914,799
Wikibase:mwapi (total)	116	165,854,510
Wikibase:label (total)	1,567,387	511,104,185
Bd:sample (total)	0	37,288
Bd:slice (total)	0	7,397
Gas:service (total)	0	16,561
Wikibase:mwapi (%)	0.00	21.63
Wikibase:label (%)	25.95	66.64
Bd:sample (%)	0.00	0.00
Bd:slice (%)	0.00	0.00
Gas:service (%)	0.00	0.00

Query distribution by graph and service (wikibase:label, wikibase:mwapi, bd:sample, gas:service). Jan 1, 2026 to Feb 27, 2026.

There is a known dependency on Gas:service by CategoriesQueryService used by Wikidata Query Service/Usage#Deepcat search. Migration work for this service will be captured in https://phabricator.wikimedia.org/T414847.

The table below reports the distribution of parameter usage of wikibase:mwapi calls on wikidata_main as a percentage of all wikibase:mwapi queries.

param_name	% of total
language	99.23%
search	97.87%
limit	3.29%
type	1.68%
srsearch	1.01%
generator	0.61%
prop	0.23%
iiprop	0.23%
gsrlimit	0.20%
gsrsearch	0.20%
pageids	0.17%
srlimit	0.04%
srnamespace	0.03%
gcmlimit	0.01%
gapnamespace	0.01%
gcmprop	0.01%
gaplimit	< 0.01%
gapfrom	< 0.01%
gcmtitle	< 0.10%
gcmtype	< 0.10%

Parameter distribution of wikibase:mwapi calls on wikidata_main Jan 1, 2026 to Feb 27, 2026.

Full details, including PII, are documented in the associated NDA Phabricator task. Superset (Presto) times out on this type of aggregation-heavy analysis, but a Jupyter notebook is available for reference at https://gitlab.wikimedia.org/repos/wikidata-platform/notebooks/-/blob/main/blazegraph-migration/T417980-label-mwapi-query-analysis.ipynb.