Jump to content

Experimentation Lab/Regulation section

From Wikitech

The regulation section allows to define the risk level of the instrument/experiment in xLab along with its Security and legal review (if needed), according to the Data Collection Guidelines.

xLab Regulation section

Note that the instrument/experiment cannot be activated until this section is fully filled. Anyway, the instrument/experiment can be registered without having defined yet the risk level that is required, and its corresponding security and legal review if needed. The risk level field can be set as Risk assessment pending and that will allow to register it (but not activate it) while you are working on defining the required details. The Regulation section can be filled properly later to set the risk level and the security and legal review (when needed). Once that is done, the instrument/experiment can be activated.

Summary of the Regulation section requirements
Risk level Security and legal review Observations
Risk assessment pending Not required The user can save the instrument/experiment but it cannot be activated
Tier 3: Low risk Not required No more requirements are needed to save or activate the instrument/experiment
Tier 2: Medium risk Required The user must provide a link the to the security and legal preview to be able to save the instrument/experiment
Tier 3: High risk Required The user must provide a link the to the security and legal preview to be able to save the instrument/experiment

In addition to the above, in the case you are registering an instrument, some privacy considerations related to the selected contextual attributes and risk level must be considered to be able to pass the validation process.

Risk level

According to the Data Collection Guidelines, there are three risk levels: Low risk, Medium risk, High risk. Note that the selected schema and contextual attributes may affect the required risk level of your instrument/experiment, because its value depends on the collected data.

When registering an instrument, the selected contextual attributes may affect directly the required risk level. There are some combinations of them that increase the required risk level. If you want to know more about this, take a look at the privacy considerations regarding contextual attributes. In addition to that, when registering and configuring your instrument, xLab will offer you some guidance and validation to help you through the process.

If you are creating an experiment using product_metrics.web_base as the stream and analytics/product_metrics/web/base as the schema, you can set the risk level as Tier 3: Low risk. That stream configuration was already reviewed by the Legal, Security, Trust and Safety team.

xLab information message to warn about the required risk level based on the selected contextual attributes

A security and legal review is needed for instruments or experiments when the risk level is set as Tier 1: High risk or Tier 2: Medium risk. A link to that review must be filled for those cases.