Metrics Platform/Implementations

This page provides an overview of Metrics Platform Client (MPC) API and behaviors, its implementations and the key differences between them. Our goal is for all MPCs to provide the same API and behave the same way, and to be as clear as possible when they do not.

At present there is a 1:1 relationship between programming language and client: effectively, the JavaScript MPC is the MediaWiki frontend client; the PHP client is the MediaWiki backend client; the Java client is the Android app client; and the Swift client is the iOS app client. In principle, however, the libraries should be language-specific but platform-agnostic, and should be able to be shared by multiple clients implemented in the same language.

All MPC implementations can be found in themetrics-platform repository.

API

`submitInteraction`

Signature

// TypeScript

interface InteractionContextData {
    action_subtype?: string;
    action_source?: string;
    action_context?: string;
}

interface MetricsClient {

    /**
     * @param streamName
     * @param schemaID The fully-qualified ID of the schema that the event
     *  ingestion service should use to validate the event
     * @param action
     * @param interactionData Additional data about the interaction, if any
     * 
     * @beta
     */
    submitInteraction: (
        streamName: string,
        schemaID: string,
        action: string,
        interactionData?: InteractionContextData
    ) => void;

}

Submits an interaction event to a stream.

An interaction event is meant to represent a basic interaction with some target or some event occurring, e.g. the user (or "performer") tapping/clicking a UI element, or an app notifying the server of its current state.

An interaction event (E) MUST validate against the /analytics/product_metrics/web/base/1.1.0 schema (or latest version number). At the time of writing, this means that E MUST have the action property and MAY have the following properties:

action_subtype
action_source
action_context

If E does not have the action property, then the event is not submitted to the stream and a warning is logged.

`submitClick`

Signature

// TypeScript

interface InteractionContextData {
    action_subtype?: string;
    action_source?: string;
    action_context?: string;
}

interface MetricsClient {

    /**
     * @param streamName
     * @param interactionData Additional data about the interaction, if any
     * 
     * @beta
     */
    submitClick: (
        streamName: string,
        interactionData?: InteractionContextData
    ) => void;

}

Submits a click event to a stream and validates it against the core interaction base schema.

This method passes its parameters to submitInteraction() with the default schema id of the web base or app base schemas, and a default action parameter of click.

`submit`

Use submitInteraction() and submitClick() when creating new instrumentation.

Signature

// TypeScript

interface MetricsClient {

    /**
     * @param streamName The destination stream
     * @param eventData The data relevant to the occurrence and specific to the
     *  instrumentation, e.g. the UI element that was clicked
     * 
     * @public
     */
    submit: (
        streamName: string,
        eventData: Record<string, any>
    ) => void;
    
}

Evaluates the stream and submitted data for submission to the Event Platform intake service according to any sampling rules specified in the stream configuration. If the event passes those sampling rules, then it is supplemented with additional metadata and submitted to the intake service.

Language-specific Notes and Key Differences

JavaScript

In the context of MediaWiki frontend development, the JavaScript MPC is provided by the EventLogging extension. It is delivered as part of the ext.eventLogging ResourceLoader module and its methods are exposed via mw.eventLog.

The JavaScript MPC and stream configurations are both delivered as part of the module and the stream configurations are not fetched again until the user navigates to another page. This minimizes the number of HTTP requests per pageview.

The EventLogging extension maintains a list of streams to be included in the module, $wgEventLoggingStreamNames, which can be used to minimize the size of the module. When $wgEventLoggingStreamNames is falsy the JavaScript MPC will not validate whether the destination stream is configured before submitting the event to the destination event service.

PHP

The PHP MPC is also part of the EventLogging extension. Its methods are exposed via MediaWiki\Extension\EventLogging\EventLogging.

Like the JavaScript MPC, when $wgEventLoggingStreamNames is falsy the PHP MPC will not validate whether the destination stream is configured before submitting the event to the destination event service.

The PHP MPC does not support sampling by pageview ID or session ID because they are not known to the server (i.e., the PHP layer of MediaWiki). Sampling support can be added if and when there is a use case for it and an appropriate sampling unit defined.

Java

The Java MPC is a standalone library that requires the instantiation of a MetricsClient object, which accepts an instance of ClientMetadata in its constructor. The ClientMetadata instance must provide app-specific getters for the values of contextual attributes. (As described above (API), these values are submitted along with whatever custom data is passed into submitMetricsEvent). The nested class method Builder.build of the enclosing MetricsClient class is responsible for instantiating an EventProcessor, a MetricsClient, and for fetching stream configurations.

Stream configurations are fetched periodically, with frequency defined by streamConfigFetchInterval in class MetricsClient, which also periodically processes the submission of queued events to the destination event intake service. If stream configurations are not yet available, queued events are held temporarily in the event queue (input buffer) per available space (as allocated in eventQueue's definition in MetricsClient, which can be overridden using method eventQueueCapacity).

If the event queue is full, events are dropped from the queue with a logged warning (from MetricsClient method submit). The metrics client will continue to send queued events to the destination event service at regular intervals (defined by sendEventsInterval). Successfully submitted events are removed from the event queue. Failed event submissions produce an error and remain in the event queue to be retried on the next queued events submission attempt.

Swift

The Swift MPC library is contained in the Event Platform group within the WMF Framework module of the Wikipedia app for iOS. The main client functionality is implemented in EventPlatformClient , along with the StorageManager and SamplingController support classes defined in separate files. Additional files in the group contain Core Data model definitions for event storage.

Stream configurations are fetched from the MediaWiki API via Meta-Wiki on app startup. If the stream configuration request fails, it is retried up to 10 times, with an increasing delay period between retries. Stream configurations are not held in persistent storage for subsequent launches.

Before stream configurations are fetched, any events received by the client are stored unconditionally in an InputBuffer with a maximum size of 128 events. If the input buffer reaches its maximum size, the oldest events are removed as needed to make room for new events. After stream configurations are loaded, events in the InputBuffer are evaluated and conditionally moved to persistent storage in Core Data, where they are held for eventual submission to the Event Platform intake service. Subsequently, the InputBuffer is no longer used, and all events received are evaluated and conditionally held in Core Data for submission.

Every 30 seconds, stale entries are pruned from the event storage table, and an attempt is made to submit all remaining entries for which submission is pending. A stale entry is defined as one that has either been successfully submitted or has existed in the storage table for more than 30 days.

The database storage model is a deviation from the Event Platform Client specification and was carried over from the previous analytics client implementation at the iOS app team's request. There is no plan to update other clients to match this behavior.

API

`submit(stream: Stream, eventData: EventInterface, domain: String? = nil)`

Types

Stream: An enum defined in EventPlatformClient that contains allowed destination stream names as values
EventInterface: EventInterface is a protocol requiring that the event data has a schema property and implements Codable

Params

domain: The wiki domain. This should be passed when the wiki domain for the current app language does not apply to the event being submitted

The EventInterface protocol will be updated to include a meta.domain property, and the domain parameter will be hard-deprecated.

API

submitInteraction

submitClick

submit

Language-specific Notes and Key Differences

JavaScript

PHP

Java

Swift

API

submit(stream: Stream, eventData: EventInterface, domain: String? = nil)

See Also

`submitInteraction`

`submitClick`

`submit`

`submit(stream: Stream, eventData: EventInterface, domain: String? = nil)`