Test Kitchen/SDK/JavaScript SDK

This page provides a guide to using the JavaScript Test Kitchen SDK for instrument and experiment creators.

Setup

To set up a local development environment for writing instrument code, follow the setup guide for MediaWiki and Test Kitchen. For more context and tooling recommendations, see the introduction to Test Kitchen development.

Create an instrument

Assuming that you have registered already your instrument in Test Kitchen UI, you can instantiate an instrument in your instrumentation code as follows:

const instrument = await mw.testKitchen.getInstrument( 'my-machine-readable-instrument-name' );

Once you've created an instrument, you can submit an event using send

Send a click event

send provides a simplified method to send any kind of event that use the Test Kitchen base schemas.

Here's an example of an instrument in JavaScript that uses Instrument#send to submit an event when a user clicks on an interwiki link.

function getLinkInteractionData( jqEvent ) {
    const link = jqEvent.target;

    return {
        action_source: link.href,
        action_context: link.title
    };
}

const instrument = await mw.testKitchen.getInstrument( 'my-machine-readable-instrument-name' );

// 'a.extiw' will match anchors that have a extiw class. extiw is used for interwiki links.
$( '#content' ).on(
    'click',
    'a.extiw',
    ( jqEvent ) => instrument.send( 'click', getLinkInteractionData( jqEvent ) )
);

The resulting event:

• includes action: click
• includes optional interaction data (action_source and action_context)
• is validated against the latest Test Kitchen base schema for web, as set in newInstrument
• is published to the specified event stream (in this case, mediawiki.product_metrics.example), as it will have been defined when registering the instrument in Test Kitchen UI

Submit an interaction event

An interaction event is meant to represent a basic interaction with some target or some event occurring. For example, a user hovers over a UI element or an app notifies the server of its current state.

Here's an example of an instrument in JavaScript that uses Instrument#send to send an event when a user hovers over an interwiki link.

$( '#content' ).on(
    'mouseover',
    'a.extiw',
    ( jqEvent ) => instrument.send( 'hover', getLinkInteractionData( jqEvent ) )
);

The resulting event:

• includes the specified value of action
• includes optional interaction data you have provided in getLinkInteractionData (action_source and action_context)
• is validated against the specified schema (in this case, the Test Kitchen base schema for web), as it will have been defined when registering the instrument in Test Kitchen UI
• is published to the specified event stream (in this case, mediawiki.product_metrics.example), as it will have been defined when registering the instrument in Test Kitchen UI

Reusable instrumentation

Both Experiment and Instrument implement mw.testKitchen.EventSenderInterface. They send analytics events and differ only in the context attached to those events.

The use() pattern

The use() method was developed to enable reusable, event-sender-agnostic instrumentation. It lets you attach generic, reusable instrumentation to an experiment or an instrument. The instrumentation is written once and then can be applied to either an experiment or an instrument without change. The instrumentation callback runs only when the user is enrolled in the experiment or is in sample for the instrument.

mw.testKitchen.getInstrument( 'my-instrument' ).use( myInstrumentation );

mw.testKitchen.getExperiment( 'my-experiment' )
   .then( ( experiment ) => experiment.use( myInstrumentation ) );

API

experiment.use( instrumentation ) / instrument.use( instrumentation )

Parameter	instrumentation - a mw.testKitchen.GenericInstrumentation callback
Returns	The same experiment/instrument (chainable)

The callback is invoked synchronously with the experiment/instrument itself typed as an EventSenderInterface:

/**
 * @param {mw.testKitchen.EventSenderInterface} eventSender
 */
function myInstrumentation( eventSender ) {
   // Set up whatever you need, then send events through `eventSender`.
   eventSender.send( 'page-visited', interactionData, contextualAttributes );
}

The only method generic instrumentation should depend on:

EventSenderInterface#send( action, interactionData, contextualAttributes )

Parameter	Type	Description
action	string	The action taken, e.g. 'click'
interactionData	Object (optional)	Additional data about the action
contextualAttributes	string[] (optional}	Per-event contextual attributes merged with those from config

When the sender is an experiment, the event is automatically decorated with experiment enrollment data before being sent.

How use() works

Depending on the sender's state, use() behaves differently:

Sender	use() behavior
Enrolled Experiment	Runs the instrumentation, sends events
UnenrolledExperiment	No-op - the instrumentation is never called
In-sample Instrument	Runs the instrumentation, sends events
UnsampledInstrument	No-op - the instrumentation is never called
OverriddenExperiment	Runs the instrumentation, but send() logs to the console

Because unenrolled/unsampled senders make use() a no-op, any setup inside your instrumentation (event listeners, mw.hook, DOM queries, etc) runs only for users who are actually enrolled in the experiment or instruments that are in-sample.

Example: Tick

For this example, we'll implement a generic "tick" instrumentation that anyone can use. The instrumentation will submit an action=tick event every n seconds.

Usage

// my-awesome-instrument.js

const tick = require( './tick.js' );

mw.testKitchen.getInstrument( 'my-awesome-instrument' ).use( tick( 1 ) );

// Or

mw.testKitchen.getExperiment( 'my-awesome-experiment' ).then(
    ( e ) => e.use( tick( 1 ) )
);

Implementation

// tick.js

/**
 * @param {Number} n The number of seconds between each action=tick event
 * @return {mw.testKitchen.GenericInstrumentation}
 */
function tick( n ) {
    return ( eventSender ) => {
        let i = 0;
        
        eventSender.send( 'tick', { action_context: String(i++) } );
        
        setInterval(
            () => {
                eventSender.send( 'tick', { action_context: String(i++) } )
            },
            n * 1000
        );
    };
};

module.exports = tick;

mw.testKitchen.getExperiment() is asynchronous and resolves with an ExperimentInterface while mw.testKitchen.getInstrument() is synchronous and returns an InstrumentInterface directly.

Chaining

use() is chainable. For example, the following snippet uses generic instrumentation for an experiment and then sends an exposure event:

mw.testKitchen.getExperiment( 'donate-experiment' )
   .then( ( experiment ) => experiment.use( ctr ).sendExposure() );

Testing

Because generic instrumentation only depends on mw.testKitchen.EventSenderInterface, you can test it in isolation (i.e. without the Test Kitchen JS SDK) by passing in a mock EventSenderInterface implementation:

QUnit.test( 'CTR instrument sends a donate-btn-click event', ( assert ) => {
    const sent = [];
    const fakeSender = {
        send: ( action, data ) => sent.push( { action, ...data } )
    };

    tick( fakeSender );

    assert.deepEqual( sent, [ { action: 'tick', action_context: '0' } ] );
} );

For integration testing that goes through getExperiment() or getInstrument, use mw.testKitchen.useFakeExperiments() or mw.testKitchen.useFakeInstruments() to stub enrollment or sampling.

use() vs. send()

use() is additive, it does not replace send(). Calling send() directly on an experiment or an instrument still works and is appropriate for one-off events. Consider use() when you have instrumentation you want to reuse across senders and/or run only when the user is enrolled in an experiment or in sample for an instrument.

Implementation

The JavaScript SDK is provided by the TestKitchen extension.

The EventLogging extension maintains a list of streams to be included in the module, $wgEventLoggingStreamNames, which can be used to minimize the size of the module. When $wgEventLoggingStreamNames is falsy the JavaScript SDK will not validate whether the destination stream is configured before submitting the event to the destination event service.

Reference

For up-to-date documentation, see the JavaScript SDK documentation microsite. For parameter descriptions and validation rules, see the web schema definition.