Metrics Platform

From Wikitech

The Metrics Platform (MP) is a suite of tools that helps Wikimedia teams make data-driven decisions about product experiences. You can use the Metrics Platform to:

  • Learn about user behavior through instrumentation, both on wiki and off wiki.
  • Change product behavior from a central location without requiring a new deployment (also known as feature flags).
  • Validate the impact of product changes through click rate, conversion rates, and other metrics.
  • Make it straightforward to implement and maintain instruments.

Create your first instrument

Preparing the environment

Instrumentation

Metrics Platform Client Libraries

Journeys

Support

If you have questions that are not covered by these pages, or additional guidance is needed, please don't hesitate to reach out. Data Products intake process is documented on wiki.

Contributing

Contributions to the Metrics Platform client libraries are most welcome and appreciated. Learn more about contributing to Metrics Platform development

Rationale and development

The Metrics Platform provides standardized algorithms, behaviors, and basic necessities for web and app instrumentation, including:

  • a predefined core interaction schema covering the most common data fields
  • a library of event schemas designed for use across a wide range of projects
  • a library of already existing instruments to be reused directly or as a starting point
  • easy means to build new schemas to further enrich events with contextual data
  • the ability to mix in different schemas depending on your needs
  • standardized session ID generation, consistent across MediaWiki, Android, and iOS
  • standardized session expiry
  • determining which events are in-sample or out-of-sample based on a specific identifier (currently: pageview, session, or app install ID).

The Metrics Platform is built on the Event Platform. A Metrics Platform client is a specialized type of Event Platform client – designed to require less work in creating instrumentation. For a summary of differences between the Metrics Platform and the Event Platform, from the code developer's perspective, see Metrics Platform/Creating An Instrument.

The development of the Metrics Platform, organized in three phases, is ongoing. The features listed above are among the results of Phase 1, and Phase 1 components may be used independently of later-phase components that are planned for development. To learn which MP components are currently available for use, see the FAQ page. The use of these components is encouraged.

Previously, different teams implemented their own analytics solutions in isolation from one another. Those solutions were typically based on the Legacy EventLogging pipeline and, more recently, the Event Platform. The Metrics Platform is an effort to unify that previous work and to establish uniformity and consistency across platforms. Its objectives include:

  • making it easier to implement and maintain instruments. The Metrics Platform will provide the APIs and protocols that MediaWiki developers need to create sophisticated instruments in as few lines of code as possible while maintaining quality, rigor, and safety.
  • making it easier for analysts to support teams which are not their primary teams. Previously, every instrument has its own quirks and conventions, which need to be remembered by analyst that works with the data produced by the instrument – including analysts that are providing temporary support.
  • making it easier to leverage data from multiple platforms to yield insights into how our users use our whole ecosystem of products in unison.

Additional objectives and benefits are listed at Metrics_Platform/FAQ#General_FAQs.

All pages