Performance
This is the entrypoint for documentation and infrastructure developed or operated by the Performance Team at Wikimedia.
For more information about the team, what we do, and how to reach us, see Wikimedia Performance Team on mediawiki.org.
Guides
The "performance practices" guides help set direction, and can be used to guide new developments, or to periodically identify areas of improvement in current code by looking for differences.
The "measure" guides help assess the actual performance of existing code, and to help iterate on proposed changes before, during, and after their development.
Tools and data
Tools we provide for use by engineers that develop or operate Wikimedia software.
Synthetic traffic
- Page drilldown (Grafana): Access results of configured synthetic testing and user journeys (split by URL, connection type, browser, and more). The same dashboard and feature-set covers Synthetic testing both from headless browsers and our Mobile Device Lab.
- ResourceLoader bundle size (Grafana): Metrics about the size of your asset bundles.
- ResourceLoader module builds (Grafana): Metrics about how long your asset bundles take to build during cache misses.
- ResourceLoader manifest size (Grafana): The size cost of your asset bundle registrations in the site-wide startup manifest.
Monitoring real traffic
- PHP Flame Graphs (performance.wikimedia.org): Flame graphs from time spent in the MediaWiki backend application (reported hourly and daily, split by service entry point).
- Navigation Timing (Grafana): Page load time and other real-user metrics from MediaWiki page views, collected via Navigation Timing and Paint Timing APIs (split by platform, country, and browser).
- responseStart by CDN host: Roundtrip latency from browsers, split by CDN server. Allows for natural experimentation and regression detection around upgrades and configuration changes to Wikimedia CDN, e.g. Linux kernel changes, and upgrades/changes to Varnish, HAProxy or ATS.
- CruX report (Grafana): Independent copy of Google's periodically published Chrome UX Report and the Core Web Vitals as measured from eligible Chrome-with-Google-account users.
- CPU benchmark (Grafana): Collected as part of our Navigation Timing beacon to help asses baseline performance. Also powers the AS Report.
- AS Report (performance.wikimedia.org): Periodic comparison of backbone connectivity from different Internet service providers, based on anonymised Navigation Timing and CPU benchmark datasets.
- WANCache (Grafana): Metrics about Memcached keys and computations in your backend code anywhere in MediaWiki core or extensions.
- Backend Pageview Timing: Backend latency from MediaWiki when generating pageviews to logged-in users (and to our CDN), split by platform.
- Save Timing (Grafana) : Time from submit to finishing the edit save on an article (and breakdown by page type, account type, and service entry point).
Debugging and development
- mw.inspect: Inspect build sizes in production or locally during development.
- Fresnel CI: Easy access to a subset of synthetic and real-user metrics during code review.
- WikimediaDebug: Capture and analyze performance profiles and debug logs in production (integrates with with Logstash and XHGui), e.g. when staging a deployment or afterwards.
- XHGui: Access to detailed per-request profiles captured via WikimediaDebug.
Misc
- noc.wikimedia.org/db.php: Database load config.
- noc.wikimedia.org/wiki.php: MediaWiki site settings.
Essays
Internal runbooks
These pages are mainly for use within the team.
Internal workflows:
- logstash.wikimedia.org (NDA restricted).
Other dashboards we regularly monitor:
Contact
- Phabricator issue tracker: #Performance-Team (triaged within 7 days)
- Libera Chat IRC: #wikimedia-perf connect
- Email: performance-team@wikimedia.org
- or reach out directly to Larissa Gaulia.