MediaWiki Engineering/Guides/Measure frontend performance

Wikimedia infrastructure

Data centers

Networking

Global traffic routing

MediaWiki SRE

MediaWiki Engineering

Multimedia

Data Engineering

Search

SRE Data Persistence

SRE Infra Foundations

SRE Observability

Wikimedia Performance

• Performance dictionary
• Web Perf Hero award 🚀
• Performance review

Guides:

• Frontend best practices
• Backend best practices
• more…

Tools:

• Arc Lamp
• AS Report
• Excimer UI
• Fresnel
• performance.wikimedia.org

Metrics:

• Metric definitions
• Real user monitoring
• User Journey
• Synthetic testing
  • Dedicated server setup
  • WebPageReplay
  • Graphite (synthetic instance)

Event Platform

Release Engineering

[]

How to measure and analyze frontend performance after you've started development. If you are starting a new project or otherwise have not yet set performance objectives, first read our guidance Frontend performance practices.

The Performance Team continuously measures site performance from both real users (via NavigationTiming) and in our synthetic lab infrastructure to find performance regressions.

This guide helps you understand the performance implications before your change reaches end-users by measuring and iterating on proposed changes before, during, and after their development.

Measure your change early

If your change will affect performance it's important to know that early in process. On your local machine you can do some tests to try find bottlenecks. Use developer tools to get in-depth information about JavaScript and layout.

It can be be hard to see things locally, then you can test your change using our speed tests directory in production and use the performance teams test infrastructure to know to verify the performance impact of your change.

Using developer tools

Using developer tools is perfect for finding in-depth information about JavaScript or CSS performance.

Chrome

Chrome has a long history of strong developer tools. You should check out the performance tab where you can investigate JavaScript and CSS performance. The best way to learn how to do this well is to watch performance audits done by Google Engineers. Check out Paul Irish investigating CNET, Time and Wikipedia or look at Sam Saccone Profiling Paint Perf or Identifying the JavaScript slowdown.

Firefox

Firefox has the Firefox Profiler https://profiler.firefox.com/docs/#/ that helps you investigate your code.

Edge

Windows Performance Toolkit is the way to test on Edge (The Microsoft team recommends that you don't use devtools because it adds too much overhead). You need to invest some time to get into the (powerful) toolkit. Let check if we can find tutorial videos as a start.

Speed tests

The Performance Team uses the speed-tests/ directory on Wikipedia.org to run performance tests and you can utilise this too. It can help you quantify the performance impact of your change before applying it to MediaWiki in production.

The way to use it is to generate a static version of a current page using tests/captureSpeedtest.php. Then you make another statics version and implement your change there. You then have a baseline test and your changed version. You can checkout how Timo created one test in https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/888105

The next step is to point the synthetic testing tools to the two new pages. If you're testing for mobile, you should use our dedicated performance device lab. To add URLs to be tested there, you need to add them to the https://gerrit.wikimedia.org/g/performance/mobile-synthetic-monitoring-tests repo. At the moment it a little bit complicated how the test runs on a phone, so please feel free to reach out to the performance-team on how you can add your tests.

Test on desktop you can add them to the file https://gerrit.wikimedia.org/r/plugins/gitiles/performance/synthetic-monitoring-tests/+/refs/heads/master/tests/desktop/desktop.txt and the next time the test runs, they will be picked.

The result will be viewable in Grafana in https://grafana.wikimedia.org/d/IvAfnmLMk/page-drilldown.

Measure using the beta cluster

The beta cluster isn't suitable for performance testing since it runs on Cloud VPS but you can use it if you want to measure the frontend performance if you combine it with our replay proxy setup.

Make sure your change exists in beta and you have a way to compare your change with the a baseline. Then you can use the performance teams infrastructure to run WebPageReplay tests with your change. For mobile you add your change in https://gerrit.wikimedia.org/g/performance/mobile-synthetic-monitoring-tests and for desktop you should use https://gerrit.wikimedia.org/r/plugins/gitiles/performance/synthetic-monitoring-tests/+/refs/heads/master/tests/desktopReplay/beta.wpr

The result will be viewable in Grafana in https://grafana.wikimedia.org/d/IvAfnmLMk/page-drilldown.

Measure a change that is already in production

Maybe your change is already in production but not default yet? For example when the Web team pushed the new vector-2022, you could run performance tests against the new version by setting the skin parameter in the URL. As long as the tests runs using a replay proxy, it doesn't matter if the data is cached or not.

To run against production and your URL to https://gerrit.wikimedia.org/g/performance/synthetic-monitoring-tests/+/refs/heads/master/tests/desktopReplay for desktop and wait for the result in Grafana in https://grafana.wikimedia.org/d/IvAfnmLMk/page-drilldown.