Incidents/20180111-LegacyEncoding/minutes
Post Mortem Meeting for train blocker - 20180111-LegacyEncoding
Date: February 16, 2018
Attendees: Mukunda, Greg, JR, Daniel, Zeljko, Victoria, Subbu
For reference:
Incident Report-https://wikitech.wikimedia.org/wiki/Incident_documentation/20180111-LegacyEncoding
Topics:
Brief Summary of Problem(s)
Changes to the revision class in which rolled out as part of 1.31.0-wmf.16 caused a serious issue on wikis with legacy encoding. The problem was caught on group1 so the problem was effectively was limited to Swedish Wiktionary.
Each revision has a flag indicating if it's using legacy encoding
How problem(s) was/were discovered
Editors on sv.wiktionary.org reported that all non-ascii characters were managed on edit <https://phabricator.wikimedia.org/T184749>
How problem(s) were introduced:
Refactoring of the revision class missed an edge case.
Learnings:
Unit tests and documentation are almost universally a good thing.
The tests themselves would not have uncovered this without a configuration change in the environment.
Ideally application would not touch global variables.
Currently you need to manually change configurations in-between test runs, due to the large number of combinations, it becomes difficult to cover.
In order to help address testing a large number of combinations, you could use https://en.wikipedia.org/wiki/All-pairs_testing
Last defense successfully caught issue, first three layers didn't
Unit test should not depend on local config. General comment is that our test should be better defined/scoped (unti test vs system test, etc...)
Parsing team's visual diff (maybe should call it pixel diffs now since visual diff is taken by VE team for a feature) testing
- That testing setup is based on https://github.com/wikimedia/integration-visualdiff which combines it with the testreduce bulk testing setup https://github.com/wikimedia/mediawiki-services-parsoid-testreduce
- The two instances where they are used are
- Comparing code changes to core (ex: replace Tidy with RemexHtml) -- https://www.mediawiki.org/wiki/Parsing/Visual_Diff_Testing
- Comparing output rendering of Parsoid to PHP parser -- https://www.mediawiki.org/wiki/Parsoid/Visual_Diffs_Testing
- Core idea: generate rendering of a wiki page via codepath 1 and another via codepath 2 and compare the 2 images. If you combine it with a bulk testing tool (can be testreduce or something else), you can run this on 10s of thousands of pages. For refactoring efforts, the diffs should be zero (or if not zero, easily explainable).
Action Items:
JR: find out if this an existing task/plan to get rid of globals.(task T189059)
Create test that tests non-ascii content.
Zeljko: run browser tests for core against beta.(task T185011)
JR: Follow up with Cindy about large scale refactoring plans currently being laid out.(task T189062)