Jump to content

Test Kitchen/Maintainer introduction

From Wikitech

This page provides an introduction to working with Test Kitchen for maintainers.

Components

As explained in Test Kitchen#About, Test Kitchen provides an standardized toolkit to create app and web instruments. This toolkit is composed of various resources, libraries or extensions:

  • MediaWiki: At this moment only MediaWiki web environment is considered in this article
  • MediaWiki Extensions: All the extensions you may need to work on Test Kitchen
  • SDKs: A set of common SDKs that allows your instrument to produce events through the EventLogging extension.
  • Schemas: A set of common data contracts to validate the events that instruments produce
  • Stream configuration: A common place where to configure the streams that instruments use to produce their events

This section aims explain all these tools to provide a technical point of view from all of them, as well as to serve as an entry point to use and setup them properly.

MediaWiki

TBD

Event Platform

TBD Event Platform

MediaWiki Extensions

EventLogging

The JavaScript Test Kitchen SDK is embedded in this extension, so its installation is a requirement to be able to create/modify any instrument or to work with Test Kitchen SDKs

EventBus

The EventBus extension propagates state changes (edit, move, delete, revision visibility, etc) to an EventGate instance, providing consumers of the service with the means of tracking changes to MediaWiki content.

EventStreamConfig

The EventStreamConfig extension is a utility extension that provides library functions and an API endpoint for exporting event stream configuration, using the variable $wgEventStreams. It does not directly provide any user functionality; rather, it provides code used by other extensions and services. It’s need for EventLogging and EventBus to work

WikimediaEvents

WikimediaEvents is the extension where almost all the instruments reside. Its installation is needed only in the case you want to create a new instrument or just to modify an existing one, except the Wikilambda instrument that has its specific extension (see below). If you only need to work on the SDKs, you don’t need to install this extension

Wikilambda

Wikilambda extension contains the instrument for Abstract Wikipedia. Its installation is needed only if you need to work on this instrument. No needed in any other case

MetricsPlatform

MetricsPlatform extension provides dynamic stream configs to EventStreamConfigs and also experiment enrollment capabilities

Dashboards

Like many other MediaWiki components, the MetricsPlatform MediaWiki extension stores information in the WANObjectCache. The maintainers of the WANObjectCache provide a parameterized dashboard of key performance metrics. The parameter is the key group used when interacting with the WANObjectCache, which is MetricsPlatform in this case. The WANObjectCache dashboard for the MetricsPlatform key group can be found here: https://grafana.wikimedia.org/d/lqE4lcGWz/wanobjectcache-key-group?orgId=1&var-kClass=MetricsPlatform

The Experiment Platform team also provide a dashboard for the key performance metrics of the MetricsPlatform extension, which are:

  • The number of requests to xLab
  • The duration of requests to xLab
  • The number of failed requests to xLab by HTTP status code

The MetricsPlatform dashboard can be found here: https://grafana.wikimedia.org/d/d1ea56fb-efc3-4f90-9cfe-618d8611c1a4/metricsplatform?orgId=1

Test Kitchen Client Libraries

Test Kitchen Client Libraries is a set of standardized client libraries for analytics instrumentation at Wikimedia. Currently supported languages include Java, PHP, JavaScript, and Swift. You don’t need it if you are just creating instruments because they can be used them through the EventLogging extension

Schemas and data contract

There are a set of predefined schemas to validate the events that instruments produce. All of them are available at https://schema.wikimedia.org but, for Metrics Platform, we only need to work with the ones included in https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-secondary

Stream Configuration

Every time we create a custom schema to produce events according to a custom stream, we’ll need to configure it. We have to provide some information to identify the stream and the schema used and some other information as sampling rate or which fields from the common schema are going to be added to the produced event

Setup

Requirements

There are no OS requirement, just Docker to run MediaWiki and all Test Kitchen dependencies as a container.

Mediawiki Docker Environment for Test Kitchen

There is a dedicated article about how to Setup Mediawiki for Test Kitchen that also includes how to produce events using Test Kitchen with the Javascript SDK through EventLogging extension to test that everything works fine and events are produced using a local Event Gate.

Specific environments per language

Regardless of whether you are going to create an instrument or working on a specific client library for Test Kitchen, you’ll need to create the right environment depending on the language/platform you are going to use. That way you’ll be able to update dependencies and run tests with the same configuration they are going to be run in the production environment. At this moment the only environment we have available is the Javascript/NodeJS one through fresh-node which is a script that creates a bash session under a container with the same configuration (OS, node, . . ) that the production environment. We are also working on something similar for PHP. Nothing similar is needed for Java and Swift, it will be enough if you install the right version of the recommended tools and libraries

Javascript

For the Javascript library we provide an environment via Fresh

A Fresh environment is a fast and ready-to-use Docker container with various developer tools pre-installed. Including Node.js, and headless browsers. It aims to help to run npm packages on your machine, without putting your personal data at risk!

To install:

curl -fsS 'https://gerrit.wikimedia.org/g/fresh/+/23.08.1/bin/fresh-install?format=TEXT' | base64 --decode | python3

Once fresh-node is installed, you can launch your NodeJS environment just running fresh-node. You'll notice that your bash prompt change because you are now running your new environment. Once there you can start installing dependencies, for example:

santi@mylaptop ~ % fresh-node
# 🌱 Fresh 23.08.1 ░ Node.js 18 ░ npm 9 ░ Firefox 102 ░ Chromium 115 ░ Debian 11 Bullseye
# image: docker-registry.wikimedia.org/releng/node18-test-browser:0.0.1
# mount: /santi      ➟ /Users/santi      (read-write)

nobody@e99bb96f8bd0:/santi$ npm install
. . .
. . .

Note: Schemas/fragments work can be also done using this environment because we use npm to build them

PHP

We are currently working on a dedicated PHP Development Environment.

In the meantime (while working on creating a PHP environment) you can run a bash session directly using the buster-php74 image to install dependencies or run tests.

For example:

docker run -it -v .:/work docker-registry.wikimedia.org/dev/buster-php74:1.1.0-s2 /bin/bash
cd /work
rm -r vendor
rm composer.lock

Java

Swift

TBD

Testing

Unit tests

TBD

End-to-end testing against a local EventGate

If you have already installed all you need to work on Test Kitchen and want to make an end-to-end test against a local EventGate to see if your installation is properly done, take a look at Test Kitchen/Local development setup.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Test_Kitchen/Maintainer_introduction&oldid=2368598#Java"