Performance/Metrics
This page documents metrics provided by Performance Team services.
These are real-user metrics (RUM) collected on a sample of Wikimedia production pageviews. The values are measuring using the W3C Navigation Timing and Paint Timing APIs that are built-in to every web browser.
To learn about how this is implemented and the journey it makes (from the MediaWiki client in your browser, all the way to Grafana) refer to our webperf-navtiming service and its documentation. They eventually end up in Graphite under the frontend.navtiming2
prefix.
Outputs:
- Grafana: Navigation Timing, visualisations with dedicated views for each metric, and with breakdowns by browser, region, and platform.
- performance.wikimedia.org, landing page with global 5-min medians.
- AS Performance Report (introduction post) web performanve metrics correlated internet providers and mobile/desktop device benchmarks.
We publish two kinds of metrics from here:
- Milestones during a page load. This is an offset from the start of the page load, thus the total duration to that instant in time.
- Durations for specific portions of a page load. These measure from the start to end of that particular operation.
Milestones
responseStart
: From navigationStart to here (Navigation Timing).domInteractive
: From navigationStart to here (Navigation Timing).domComplete
: From navigationStart to here (Navigation Timing).loadEventStart
: From navigationStart to here (Navigation Timing).loadEventEnd
: From navigationStart to here (Navigation Timing). Also known as "page load time (PLT)" or "On load", which typically corresponds with the browser's page loading indicator.firstPaint
: From navigationStart to first-timing (Paint Timing).firstContentfulPaint
: From navigationStart to first-contentful-paint (Paint Timing).mediaWikiLoadEnd
: From navigationStart to when all JavaScript code queued to load on a page has arrived and finished its initial script execution. This is analogous to when themw.loader.using(RLPAGEMODULES).then()
Promise resolves.
Durations
dns
: Computed asdomainLookupEnd - domainLookupStart
, our intermediary layer labels this "dnsLookup".unload
: Computed asunloadEventEnd - unloadEventStart
.redirect
: Computed asredirectEnd - redirectStart
, our intermediary layer labels this "redirecting".tcp
: Computed asconnectEnd - connectStart
. (As per the spec, browsers include the TLS handshake for HTTPS).ssl
: Computed asconnectEnd - secureConnectionStart
. (As per the spec, browsers report this as subset oftcp
).request
: Computed asresponseStart - requestStart
.response
: Computed asresponseEnd - responseStart
.processing
: Computed asdomComplete - responseEnd
.onLoad
: Computed asloadEventEnd - loadEventStart
.
See also
See phab:T104902 for how we validate incoming data:
- We don't filter out the value 0 (zero).
- When negative numbers are encountered due to browser bugs, we reject the entire beacon, not just that one data point for that one metric. This ensures fair comparison between two metrics by representing the same pageview samples. We measure how often this happens in Grafana: EventLogging-schema / NavigationTiming via the
nonCompliant
counter. The details of this are logged by webperf-navtiming tojournalctl
.
- Milestone diagram: https://www.w3.org/TR/resource-timing-2/#attribute-descriptions
Save Timing
We define two metrics to represent the duratation of time to save of an edit. To save an edit, in MediaWiki, means to create or change a wiki page.
Backend Save Timing
Backend Save Timing measures time spent in MediaWiki PHP, from the process start (REQUEST_TIME_FLOAT
) until the response is flushed to the web server for sending to the client (PRESEND
). The instrumentation resides in the WikimediaEvents extension (source), and is published to Graphite under MediaWiki.timing.editResponseTime
.
The metric is plotted in Grafana: Backend Save Timing Breakdown, and includes slices by account type (bot vs human), by entry point (index.php wikitext editor, vs api.php for VisualEditor and bots), and by page type or namespace (Wikipedia content, or Wikidata entity, or discussion pages).
Frontend Save Timing
Frontend Save Timing is measured as time from pressing "Publish changes" from a user interface in a web browser (e.g. submitting the edit page form) until that browser recieves the first byte of the server response that will render the confirmation page (e.g. the article with their edit applied and a "Post-edit" message).
This is implemented as navigationStart
(the click to submit the form over HTTP POST) to responseStart
(the first byte after the server has finished processing the edit, redirected, and responded to the subsequent GET).
Instrumented by Extension:NavigationTiming (client source code), processed webperf-navtiming, and published to Statsd/Graphite under the mw.performance.save
.
The metric is plotted toward the bottom of Grafana: Save Timing, and includes slices by wiki (group1 is Wikidata/Commons, group2 is Wikipedia, per Train groups).
See also
When investigating Save Timing metrics, it may be useful to correlate with:
- performance.wikimedia.org: Flame Graphs, which shows where in the MW codebase time is spent code during particular operations. Use the "api" graph for edits via the API, and the "fn-EditAction" graph for edits via index.php.
- Grafana: Application Servers RED , which measures the overall load and throughput of our web servers by cluster (e.g. appservers for index.php, api_appservers for api.php) and by method (e.g. POST is mostly edits).
- Grafana: Edit Count, which measures the overall rate of edits being saved. This count is derived directly from the Backend Save Timing metric, and thus corresponds fully. Any edit save that we measure for backend save timing is counted here, and vice-versa.
Synthetic metrics
The synthetic metrics are collected using VisualMetrics analyzing a video recording of the screen when the page is loading.
WebPageTest
The WebPageTest metrics are available in Graphite under the webpagetest
prefix.
Visual Metrics
render
: The time when something for the first time is painted within the viewport.BackgroundImage
: The time when the largest background image is painted at its final position within the viewport.Heading
: The time when the first h1/h2 heading is painted at its final position within the viewport.LargestImage
: The time when the largest image is painted at its final position within the viewport.SpeedIndex
: SpeedIndex is the average time at which visible parts of the page are displayed. It is expressed in milliseconds and dependent on size of the view port.lastVisualChange
: The time when the last paint happens within the viewport.VisualComplete85
: The time when 85% of the content within the viewport is paintedVisualComplete95
: The time when 95% of the content within the viewport is painted.VisualComplete99
: The time when 99% of the content within the viewport is painted.
Other timing metrics
fullyLoaded
: The time all requests on the page has finished loading.TTFB
: The time to first byte delivered from the server.domComplete
: domComplete from the Navigation Timing API.
Size and requests
WebPageTest also collects the number of bytes/request per type. Add .bytes
or .requests
to the type to get that information.
html
css
js
flash
font
video
image
total
: The total amount of bytes/requests for the tested page.
CPU times
These metrics are Chrome only.
Idle
: Time spent being idle.Layout
: Time spent rendering the screen.Painting
: Time spent in painting the screen.Scripting
: Time spent running JavaScript.Loading
: Time spent in loading assets.
Visual Metrics
render
: The time when something for the first time is painted within the viewport.BackgroundImage
: The time when the largest background image is painted at its final position within the viewport.Heading
: The time when the first h1/h2 heading is painted at its final position within the viewport.LargestImage
:The time when the largest image is painted at its final position within the viewport.SpeedIndex
:The Speed Index is the average time at which visible parts of the page are displayed. It is expressed in milliseconds and dependent on size of the view port.lastVisualChange
:The time when the last paint happens within the viewport.VisualComplete85
::The time when 85% of the content within the viewport is painted.VisualComplete95
:The time when 95% of the content within the viewport is painted.VisualComplete99
:The time when 99% of the content within the viewport is painted.
Other timing metrics
fullyLoaded
:The time all requests on the page has finished loading.TTFB
: The time to first byte delivered from the server.domComplete
: domComplete from the Navigation Timing APi.
Size and requests
WebPageTest also collects the number of bytes/request per type. Add .bytes
or .requests
to the type to get that information.
html
css
js
flash
font
video
image
total
: The total amount of bytes/requests for the tested page.
CPU times
These metrics are Chrome only.
Idle
:Time spent being idle.Layout
:Time spent rendering the screen.Painting
: Time spent in painting the screen.Scripting
:Time spent running JavaScript.Loading
:Time spent in loading assets.
WebPageReplay
The WebPageReplay metrics are available in graphite-synthetic-testing
under the sitespeed_io
prefix (it's Browsertime/sitespeed.io that collect the metrics).
At the moment we all the default metrics that is collected when you sitespeed.io. Here's a list of some of the most important ones:
Visual Metrics
FirstVisualChange
: The time when something for the first time is painted within the viewport.Heading
: The time when the first h1 heading is painted at its final position within the viewport.LargestImage
: The time when the largest image is painted at its final position within the viewport.VisualComplete85
: The time when 85% of the content within the viewport is painted.VisualComplete95
: The time when 95% of the content within the viewport is painted.VisualComplete99
: The time when 99% of the content within the viewport is painted.LastVisualChange
: The time when the last paint happens within the viewport.SpeedIndex
: The Speed Index is the average time at which visible parts of the page are displayed. It is expressed in milliseconds and dependent on size of the view port.PerceptualSpeedIndex
: The Perceptual Speed Index is sensitive to page elements moving around (that is not Speed Index). For example if a campaign/banner pushes the elements, that will have bigger impact on the perceptual metric.
CPU metrics
LongTasks.tasks
: the number CPU long tasks (> 50 ms)LongTasks.beforeFirstPaint
: the number CPU long tasks happening before anything is painted on the screen (> 50 ms)LongTasks.totalDuration
: the total time (in ms) spent in CPU long tasksLongTasks.beforeFirstPaint.totalDuration
: the total time (in ms) spent in CPU long tasks before anything was painted on the screen.