mw-experimental

mw-experimental is a MediaWiki On Kubernetes deployment in WikiKube, for ad-hoc testing via WikimediaDebug.

Design

Each data center runs one mw-experimental pod from a dedicated wikikube-worker node, reachable to you by SSH.

These mw-experimental pods run the same mediawiki-multiversion-debug docker images as mwdebug pods, but with MediaWiki code mounted from the host's /srv/mediawiki directory, rather than the code found in the container. This allows you to edit the files for ad-hoc testing.

This service replaces mwdebug (more info on task T276994)

Using the service

SSH to the Deployment server and recreate the mw-experimental pod in Kubernetes for your chosen data center. This will apply the latest images, matching the rest of production mw-on-k8s. Scap skips restarting/recreating mw-experimental pods during deployments. While code under /srv/mediawiki is updated dynamically, the underlying docker image may be several days behind on infrastructure changes until you redeploy it (e.g. Puppet, Debian packages, Apache config, PHP settings):
```
deploy:$ cd /srv/deployment-charts/helmfile.d/services/mw-experimental
deploy:$ helmfile -e eqiad -i apply

or 

deploy:$ helmfile -e codfw -i apply
```
SSH to a host mw-experimental.eqiad.wmnet or mw-experimental.codfw.wmnet
There is timer updating /srv/mediawiki every hour. You can check when it last ran:
```
systemctl status mw-experimental-mediawiki-image-update.timer
```

Despite this, if you want to update it again, you may run the timer on demand (takes ~3mins):

wikikube-worker-exp1001:~$ sudo systemctl restart mw-experimental-mediawiki-image-update.service

Make your changes in /srv/mediawiki as usual. Files are editable as any user in the deployment group; changes to PHP files should take effect immediately (no need to restart php-fpm or anything like that).
Go to WikimediaDebug and choose k8s-mw-experimental-eqiad or k8s-mw-experimental-codfw

Notes:

We have a timer updating /srv/mediawiki every hour

To prevent code refreshing, you may run

wikikube-worker-exp1001:~$ /usr/local/sbin/mw-experimental-mediawiki-image-update --lock
 
wikikube-worker-exp1001:~$ /usr/local/sbin/mw-experimental-mediawiki-image-update --unlock

If you would like to discard your changes quickly and start over, you do not need to run helmfile again, unless some breaking changes have been deployed in the meantime. Restarting mw-experimental-mediawiki-image-update.service should be enough.
Image versions used on mw-experimental can be found either on the experimental hosts themselves or the deployment server under /etc/helmfile-defaults/mediawiki/release/mw-experimental-pinkllama.yaml

Dashboards

Same url as before on logstash
mw-experimental on grafana

Known limitations and workarounds

Checking out a Gerrit change

Τhere is no straightforward way to check out a certain Gerrit change. Unofficial workaround: do some Git trickery to turn the directories into Git repositories.

cd /srv/mediawiki/php-1.46.0-wmf.26/extensions/cldr/
sudo -u mwdeploy git init -b wmf/1.46.0-wmf.26
sudo -u mwdeploy git remote add gerrit https://gerrit.wikimedia.org/r/mediawiki/extensions/cldr.git
sudo -u mwdeploy git fetch --depth=1 gerrit wmf/1.46.0-wmf.26
sudo -u mwdeploy git update-ref HEAD gerrit/wmf/1.46.0-wmf.26
sudo -u mwdeploy git reset

(Substitute the right directories, branches and Gerrit URLs as needed.) Then you can fetch the Gerrit change from the refs/changes namespace, see Gerrit documentation or mw:Gerrit/Advanced usage, and check it out at FETCH_HEAD.

If you break anything, revert your changes with sudo systemctl restart mw-experimental-mediawiki-image-update.service.

Design

Using the service

Dashboards

Known limitations and workarounds

Checking out a Gerrit change

See also