Performance/Metrics

From Wikitech
Jump to navigation Jump to search

This page documents metrics provided by Performance Team services.

navtiming

These are real-user metrics (RUM) collected on a sample of Wikimedia production pageviews. The values are measuring using the W3C Navigation Timing and Paint Timing APIs that are built-in to every web browser.

To learn about how this is implemented and the journey it makes (from the MediaWiki client in your browser, all the way to Grafana) refer to our webperf-navtiming service and its documentation. They eventually end up in Graphite under the frontend.navtiming2 prefix.

Outputs:

We publish two kinds of metrics from here:

  • Milestones during a page load. This is an offset from the start of the page load, thus the total duration to that instant in time.
  • Durations for specific portions of a page load. These measure from the start to end of that particular operation.

Milestones

  • responseStart: From navigationStart to here (Navigation Timing).
  • domInteractive: From navigationStart to here (Navigation Timing).
  • domComplete: From navigationStart to here (Navigation Timing).
  • loadEventStart: From navigationStart to here (Navigation Timing).
  • loadEventEnd: From navigationStart to here (Navigation Timing). Also known as "page load time (PLT)" or "On load", which typically corresponds with the browser's page loading indicator.
  • firstPaint: From navigationStart to first-timing (Paint Timing).
  • firstContentfulPaint: From navigationStart to first-contentful-paint (Paint Timing).
  • mediaWikiLoadEnd: From navigationStart to when all JavaScript code queued to load on a page has arrived and finished its initial script execution. This is analogous to when the mw.loader.using(RLPAGEMODULES).then() Promise resolves.

Durations

  • dns: Computed as domainLookupEnd - domainLookupStart, our intermediary layer labels this "dnsLookup".
  • unload: Computed as unloadEventEnd - unloadEventStart.
  • redirect: Computed as redirectEnd - redirectStart, our intermediary layer labels this "redirecting".
  • tcp: Computed as connectEnd - connectStart. (As per the spec, browsers include the TLS handshake for HTTPS).
  • ssl: Computed as connectEnd - secureConnectionStart. (As per the spec, browsers report this as subset of tcp).
  • request: Computed as responseStart - requestStart.
  • response: Computed as responseEnd - responseStart.
  • processing: Computed as domComplete - responseEnd.
  • onLoad: Computed as loadEventEnd - loadEventStart.

See also

See phab:T104902 for how we validate incoming data:

  • We don't filter out the value 0 (zero).
  • When negative numbers are encountered due to browser bugs, we reject the entire beacon, not just that one data point for that one metric. This ensures fair comparison between two metrics by representing the same pageview samples. We measure how often this happens in Grafana: EventLogging-schema / NavigationTiming via the nonCompliant counter. The details of this are logged by webperf-navtiming to journalctl.

Save Timing

We define two metrics to represent the duratation of time to save of an edit. To save an edit, in MediaWiki, means to create or change a wiki page.

Backend Save Timing

Backend Save Timing measures time spent in MediaWiki PHP, from the process start (REQUEST_TIME_FLOAT) until the response is flushed to the web server for sending to the client (PRESEND). The instrumentation resides in the WikimediaEvents extension (source), and is published to Graphite under MediaWiki.timing.editResponseTime.

The metric is plotted in Grafana: Backend Save Timing Breakdown, and includes slices by account type (bot vs human), by entry point (index.php wikitext editor, vs api.php for VisualEditor and bots), and by page type or namespace (Wikipedia content, or Wikidata entity, or discussion pages).

Frontend Save Timing

Frontend Save Timing is measured as time from pressing "Publish changes" from a user interface in a web browser (e.g. submitting the edit page form) until that browser recieves the first byte of the server response that will render the confirmation page (e.g. the article with their edit applied and a "Post-edit" message).

This is implemented as navigationStart (the click to submit the form over HTTP POST) to responseStart (the first byte after the server has finished processing the edit, redirected, and responded to the subsequent GET).

Instrumented by Extension:NavigationTiming (client source code), processed webperf-navtiming, and published to Statsd/Graphite under the mw.performance.save.

The metric is plotted toward the bottom of Grafana: Save Timing, and includes slices by wiki (group1 is Wikidata/Commons, group2 is Wikipedia, per Train groups).

See also

When investigating Save Timing metrics, it may be useful to correlate with:

  • performance.wikimedia.org: Flame Graphs, which shows where in the MW codebase time is spent code during particular operations. Use the "api" graph for edits via the API, and the "fn-EditAction" graph for edits via index.php.
  • Grafana: Application Servers RED , which measures the overall load and throughput of our web servers by cluster (e.g. appservers for index.php, api_appservers for api.php) and by method (e.g. POST is mostly edits).
  • Grafana: Edit Count, which measures the overall rate of edits being saved. This count is derived directly from the Backend Save Timing metric, and thus corresponds fully. Any edit save that we measure for backend save timing is counted here, and vice-versa.

Synthetic metrics

The synthetic metrics are collected using VisualMetrics analyzing a video recording of the screen when the page is loading.

WebPageTest

The WebPageTest metrics are available in Graphite under the webpagetest prefix.

Visual Metrics

  • render: The time when something for the first time is painted within the viewport.
  • BackgroundImage: The time when the largest background image is painted at its final position within the viewport.
  • Heading: The time when the first h1/h2 heading is painted at its final position within the viewport.
  • LargestImage: The time when the largest image is painted at its final position within the viewport.
  • SpeedIndex: SpeedIndex is the average time at which visible parts of the page are displayed. It is expressed in milliseconds and dependent on size of the view port.
  • lastVisualChange: The time when the last paint happens within the viewport.
  • VisualComplete85: The time when 85% of the content within the viewport is painted
  • VisualComplete95: The time when 95% of the content within the viewport is painted.
  • VisualComplete99: The time when 99% of the content within the viewport is painted.

Other timing metrics

  • fullyLoaded: The time all requests on the page has finished loading.
  • TTFB: The time to first byte delivered from the server.
  • domComplete: domComplete from the Navigation Timing API.

Size and requests

WebPageTest also collects the number of bytes/request per type. Add .bytesor .requeststo the type to get that information.

  • html
  • css
  • js
  • flash
  • font
  • video
  • image
  • total: The total amount of bytes/requests for the tested page.

CPU times

These metrics are Chrome only.

  • Idle: Time spent being idle.
  • Layout: Time spent rendering the screen.
  • Painting: Time spent in painting the screen.
  • Scripting: Time spent running JavaScript.
  • Loading: Time spent in loading assets.

Visual Metrics

  • render: The time when something for the first time is painted within the viewport.
  • BackgroundImage: The time when the largest background image is painted at its final position within the viewport.
  • Heading: The time when the first h1/h2 heading is painted at its final position within the viewport.
  • LargestImage:The time when the largest image is painted at its final position within the viewport.
  • SpeedIndex:The Speed Index is the average time at which visible parts of the page are displayed. It is expressed in milliseconds and dependent on size of the view port.
  • lastVisualChange:The time when the last paint happens within the viewport.
  • VisualComplete85::The time when 85% of the content within the viewport is painted.
  • VisualComplete95:The time when 95% of the content within the viewport is painted.
  • VisualComplete99:The time when 99% of the content within the viewport is painted.

Other timing metrics

  • fullyLoaded:The time all requests on the page has finished loading.
  • TTFB: The time to first byte delivered from the server.
  • domComplete: domComplete from the Navigation Timing APi.

Size and requests

WebPageTest also collects the number of bytes/request per type. Add .bytes or .requests to the type to get that information.

  • html
  • css
  • js
  • flash
  • font
  • video
  • image
  • total: The total amount of bytes/requests for the tested page.

CPU times

These metrics are Chrome only.

  • Idle:Time spent being idle.
  • Layout:Time spent rendering the screen.
  • Painting: Time spent in painting the screen.
  • Scripting:Time spent running JavaScript.
  • Loading:Time spent in loading assets.

WebPageReplay

The WebPageReplay metrics are available in graphite-synthetic-testing under the sitespeed_io prefix (it's Browsertime/sitespeed.io that collect the metrics).

At the moment we all the default metrics that is collected when you sitespeed.io. Here's a list of some of the most important ones:

Visual Metrics

  • FirstVisualChange: The time when something for the first time is painted within the viewport.
  • Heading: The time when the first h1 heading is painted at its final position within the viewport.
  • LargestImage: The time when the largest image is painted at its final position within the viewport.
  • VisualComplete85: The time when 85% of the content within the viewport is painted.
  • VisualComplete95: The time when 95% of the content within the viewport is painted.
  • VisualComplete99: The time when 99% of the content within the viewport is painted.
  • LastVisualChange: The time when the last paint happens within the viewport.
  • SpeedIndex: The Speed Index is the average time at which visible parts of the page are displayed. It is expressed in milliseconds and dependent on size of the view port.
  • PerceptualSpeedIndex: The Perceptual Speed Index is sensitive to page elements moving around (that is not Speed Index). For example if a campaign/banner pushes the elements, that will have bigger impact on the perceptual metric.

CPU metrics

  • LongTasks.tasks: the number CPU long tasks (> 50 ms)
  • LongTasks.beforeFirstPaint: the number CPU long tasks happening before anything is painted on the screen (> 50 ms)
  • LongTasks.totalDuration: the total time (in ms) spent in CPU long tasks
  • LongTasks.beforeFirstPaint.totalDuration: the total time (in ms) spent in CPU long tasks before anything was painted on the screen.
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Performance/Metrics&oldid=2060915"