Round trip testing
Kick off a round of rt-testing with this command.
$ bin/start-rt-test.sh <uid> <git-sha>
To check the test results visit the web frontend to the results. If there are unexpected failures vendor patch should be blocked until team is confident to release new code.
Occasionally, mysql would have crashed. If so, log on to testreduce1001.eqiad.wmnet and
sudo service mysql restart && sudo service parsoid-rt restart
Other services, like the web frontend to the results, are also sometimes down and would need a similar restart. It's also a good practice to verify that a run isn't currently in progress before you get started, and then announcing to the team when you're kicking one off.
Freeing disk space
The start-rt-test.sh script should alert when disk space needs freeing.
mysql on testreduce1001 is mounted on /srv/data which has 50GB of disk space and this fills up fast. So, every few weeks, you may need to run
nohup ~ssastry/purge_old_results.sh on testreduce1001. T296051 is still pending which will increase disk space but doesn't fundamentally eliminate the need to run this script periodically, say every X weeks. If df shows usage at or above 90%, you probably should run this script.
This script takes about 90 mins to complete.
Visit https://www.mediawiki.org/wiki/Parsoid/Round-trip_testing for more of the details
Preparing a new (weekly) Parsoid deploy
See the main page for all the details!
Reviewing the vendor patch
- Its almost always a routine review with +2
- Usual checks
composer.jsonhas the right version bump
composer.lockdoesn't introduce any unnecessary changes other than the parsoid patch
- version numbers related to parsoid, content hash values, timestamps
- Ensure that no files are missing
- patches added new files but vendor patch didn't include them
git review -d <vendor-patch-id>and
composer update --no-devdoesn't introduce changes
- If the patch upload hasn't provided info about the following (either on the patch or on Slack):
- Make sure that after +2 patch is actually merged
For example from a patch:
- rt-testing results are clean
- parsoid-tests logstash doesn't have errors
- we have a deployment log on mediawiki.org
- reviewed the patches on the deploy log for potential issues with train rollback and there are no issues
Check Production logs
- Check the Parsoid dashboard in logstash regularly and look for any crashers or other issues and file phabricator tasks as needed.
- You can use the Phatality feature to submit tasks right from logstash.
- Expand the logstash entry and click on the Phatality tab.
- Click on the 'desc' field to copy it since it isn't submitted automatically.
- Click 'Submit' and paste the copied 'desc' field into the Description box.
- Make any necessary tweaks to the title and project tags (ex: add Parsoid at the very least, maybe Content-Transform-Team as well and Parsoid-Read-Views if you already know for a fact that this is important to fix for read views).
- (OPTIONAL): As the train rolls out, checking the recent changes feeds for corruption is encouraged, though when we deployed the service independently outside the train, we used to do this immediately after deploy.
- Subbu: FWIW, I haven't done this in more than 18 months.
Visual diff testing
You need to log on to parsing-qa-02.wikitextexp.eqiad1.wikimedia.cloud for this.
$ vi /etc/testreduce/parsoid-vs-core-vd-client.config.js ... edit gitCommitFetch function to use a suitable test id ... $ sudo service parsoid-vs-core-vd-client restart $ sudo journalctl -f -u parsoid-vs-core-vd ... watch logs and restart service if necessary (usually never) ... ... visit http://parsoid-vs-core.wmflabs.org/ to see results ...
There are multiple visual diff instances that exist on parsing-qa-02. Find the other configs in /etc/testreduce there.
You can also switch databases in /etc/testreduce/parsoid-vs-core-vd.settings.js (either parsoid_vs_core or parsoid_vs_core_talkns if you want to run tests with different test sets. As the name implies, parsoid_vs_core_talkns only has test pages from the talk namespace.
Publishing results on mediawiki
bash ~ssastry/stats.sh > /tmp/stats to generate a wikitext table to pop onto any wiki page. You will have to suitably the database (testset) that you want to generate results for. Example output on https://www.mediawiki.org/wiki/Parsing/Parser_Unification/Pixel_Diff_Testing_Stats
T295907 tracks the task of puppetizing the config on parsing-qa-02
Visit https://www.mediawiki.org/wiki/Parsing/Visual_Diff_Testing for more of details for visual diff testing