XKey

From Wikitech
Jump to: navigation, search
Wikimedia infrastructure

Data centres and PoPs

Networking

HTTP Caching


Wiki

Media

Logs

The xkey Varnish module adds secondary hashes to objects, allowing fast purges of objects with the same key.

Setting Object Keys

Keys should preferably be set by the application in the xkey response header. Multiple keys can be specified per header line with a space separator:

HTTP/1.1 OK
Server: Apache/2.2.15
xkey: 8155054 166412 234323

or in multiple xkey response headers:

HTTP/1.1 OK
Server: Apache/2.2.15
xkey: 8155054
xkey: 166412
xkey: 234323k

Alternatively, keys can also be set in VCL:

sub vcl_backend_response {
    set beresp.http.xkey = "8155054 166412 234323";
}

Scaling

We currently do not support setting the same XKey on very large numbers of objects. In practice something on the order of 1-100 objects attached to a given XKey is reasonable. It is not reasonable to use XKeys which would be relevant to hundreds of objects (or more), such as using XKeys to indicate broad references from templates, transclusions, backlinks, etc. Primarily we plan to have XKeys indicating unique objects such as wiki articles or image filenames, and shared between e.g. all of their direct representations (article: desktop HTML, mobile HTML, RESTBase HTML; various thumbnailed sizes of an image, etc) and metadata (?action=history of an article, etc).

Namespacing and Encoding

Because (slight) over-purging on namespacing and encoding collisions aren't really a pragmatic issue, there's no current plan to define a rigorous meta-schema like /v1/mw/enwiki/title/Foo. For wiki articles, the XKey should be the wiki's official database name prefixed and suffixed with forward slashes (e.g. /enwiki/), and then the article title. For images and thumbnails, the XKey should be the original image pathname on the upload servers. There is no special prefix when sent as an `XKey:` header in an HTTP response. When used for purging, XKey values should be prefixed with xkey: to differentiate them from raw https URL purges.

XKey values should be percent-encoded according to normal HTTP URI rules. XKey values should not encode characters in the Unreserved Set, may encode characters in the Reserved Set, and must encode all other characters. This encoding will be normalized on reception for both XKey header output and PURGE requests as best as possible, but not all application encoding errors can be corrected (especially under- or over- encoding of literal %). It would be best to ensure the application enforces sane encoding to avoid issues.

Examples

MediaWiki article

cache_upload images