Metrics Platform/Implementations

This page provides an overview of Metrics Platform Client (MPC) API and behaviors, its implementations and the key differences between them. Our goal is for all MPCs to provide the same API and behave the same way, and to be as clear as possible when they do not.

At present there is a 1:1 relationship between programming language and client: effectively, the JavaScript MPC is the MediaWiki frontend client; the PHP client is the MediaWiki backend client; the Java client is the Android app client; and the Swift client is the iOS app client. In principle, however, the libraries should be language-specific but platform-agnostic, and should be able to be shared by multiple clients implemented in the same language.

At the time of writing, all MPCs can be found in the libs/metrics-platform repository.

API

submit( streamName: string, eventData )

Evaluates the stream and submitted data for submission to the Event Platform intake service according to any sampling rules specified in the stream configuration. If the event passes those sampling rules, then it is supplemented with additional metadata and submitted to the intake service.

Params

streamName

The destination stream name. If the MPC does not have the configuration for the stream available, then the call fails and the event is discarded

eventData

The event data

dispatch( eventName: string, ?customData )

Determines which streams have registered interest in the event with the given name and for each stream:

1. An event E is constructed with:
   • E.$schema set to /analytics/mediawiki/client/metrics_event/1.1.0
   • E.name set to eventName
   • E.dt set
2. If customData is not null, then for each key, value in customData:
   1. Assert that key is snake case
   2. Create a map M with:
      1. M.data_type set to the type of customData[key]
      2. M.value to the string representation of customData[key]
   3. Set E.custom_data[key] to M
3. E is supplemented with additional contextual data according to the stream configuration
4. E is filtered according to any filtering rules specified in the stream configuration. If E does not pass all those filtering rules, then it is discarded
5. Submit E to the stream using submit( S, E )

Note well that all events are submitted with the value for dt.

Params

eventName

The name of the event, e.g. pageview, web.ui.click

customData

The data relevant to the occurrence and specific to the instrumentation, e.g. the UI element that was clicked

No stream configs

Note that if no stream configs for a given event are found, the event will not be dispatched nor submitted.

Notes and Key Differences

JavaScript

In the context of MediaWiki frontend development, the JavaScript MPC is provided by the EventLogging extension. It is delivered as part of the ext.eventLogging ResourceLoader module and its methods are exposed via mw.eventLog.

The JavaScript MPC and stream configurations are both delivered as part of the module and the stream configurations are not fetched again until the user navigates to another page. This minimizes the the number of HTTP requests per pageview.

The EventLogging extension maintains a list of streams to be included in the module, $wgEventLoggingStreamNames, which can be used to minimize the size of the module. When $wgEventLoggingStreamNames is falsy the JavaScript MPC will not validate whether the destination stream is configured before submitting the event to the destination event service.

PHP

The PHP MPC is also part of the EventLogging extension. Its methods are exposed via MediaWiki\Extension\EventLogging\EventLogging.

Like the JavaScript MPC, when $wgEventLoggingStreamNames is falsy the PHP MPC will not validate whether the destination stream is configured before submitting the event to the destination event service.

The PHP MPC does not support sampling by pageview ID or session ID because they are not known to the server. Sampling support can be added if and when there is a use case for it and an appropriate sampling unit defined.

Swift

The Swift EPC library is contained in the Event Platform group within the WMF Framework module of the Wikipedia app for iOS. The main client functionality is implemented in EventPlatformClient , along with the StorageManager and SamplingController support classes defined in separate files. Additional files in the group contain Core Data model definitions for event storage.

Stream configurations are fetched from the MediaWiki API via Meta-Wiki on app startup. If the stream configuration request fails, it is retried up to 10 times, with an increasing delay period between retries. Stream configurations are not held in persistent storage for subsequent launches.

Before stream configurations are fetched, any events received by the client are stored unconditionally in an InputBuffer with a maximum size of 128 events. If the input buffer reaches its maximum size, the oldest events are removed as needed to make room for new events. After stream configurations are loaded, events in the InputBuffer are evaluated and conditionally moved to persistent storage in Core Data, where they are held for eventual submission to the Event Platform intake service. Subsequently, the InputBuffer is no longer used, and all events received are evaluated and conditionally held in Core Data for submission.

Every 30 seconds, stale entries are pruned from the event storage table, and an attempt is made to submit all remaining entries pending submission. A stale entry is defined as one that has either been successfully submitted or has existed in the storage table for more than 30 days.

API

submit(stream: Stream, eventData: EventInterface, domain: String? = nil)

Types

Stream

An enum defined in EventPlatformClient that contains allowed destination stream names as values

EventInterface

EventInterface is a protocol requiring that the event data has a schema property and implements Codable

Params

domain

The wiki domain. This should be passed when the wiki domain for the current app language does not apply to the event being submitted

See Also

• See https://github.com/linehan/wmf-epc for early prototype implementations of the JavaScript, Java, and Swift MPCs