Performance/WebPageReplay/Alerts
We use Grafana alerts for WebPageReplay tests to get notified of potential frontend performance regressions, based on the metrics we collect from the Browsertime and WebPageReplay setup.
We run 5 runs per URL against WebPageReplay and record a video in 30 fps and take the median metrics. Since we run against a local proxy (WebPageReplay) we set the latency to 100ms to slowdown the rendering.
We have alerts for FirstVisualChange that will fire if there's a 20-40 ms regression all the three pages tested for that wiki. We run tests using Firefox and Chrome for desktop and emulated mobile Chrome for mobile.
At the moment we test three pages on each of these wikis and on their corresponding mobile wiki:
The main test repo hold the URLs for all pages tested, except the enwiki URLs.
Mobile test URLs and enwiki mobile URLs.
You find all the alerts under Performance Team Alerts in Grafana.