Metrics Platform/Measure product health

Product health metrics are key performance indicators for measuring and monitoring the impact of a product. This page describes how to measure product health using the Metrics Platform.

This guide describes the process for measuring product health using the Metrics Platform Experimentation Lab (also known as MPIC). For the manual process for creating an instrument, see Create an instrument. If you're unsure which process to use, contact Data Platform.

Plan

Measurement plan

Instruments collect data about user interactions so that we can answer questions about product experiences. Before you can start collecting data, create a measurement plan (template) that documents what data you plan to collect, why, and how you plan to analyze the data.

You can write your measurement plan in a document or on a Phabricator task, depending on the scale of the project. For examples of measurement plans, see the folder on Google Drive.

Instrumentation spec

Once you have a measurement plan, the next step is to create an instrumentation specification (template). The instrumentation spec defines all the data you'll collect for your instrument. The spec is also a useful tool for engineers to ensure that all events are being produced and received correctly. For a template and examples of instrumentation specs, see the folder on Google Drive. For more information about designing an instrumentation spec, see the instrument guide.

Data collection guidelines

All data collection activities must follow the data collection guidelines. Once you've identified the applicable risk tier, you can use your measurement plan and instrumentation spec to complete the steps in the guidelines under "What should WMF teams do next?".

Code

You can write your instrument code in the WikimediaEvents extension or in your product codebase. See the API docs to learn how to code an instrument.

Launch

Once your instrument code has been deployed to production, complete the launch-your-baseline-instrument form to configure your instrument and start collecting data.

Monitor

You can use the experimentation directory to monitor the progress of your instrument. Your event data will be available in a Hive table with the same name as the event stream, with dots and dashes replaced with underscores.

Manage

The experimentation directory provides options to edit and turn off your instrument using the Actions menu.