Incident documentation/2017-02-22 www-portals
Jump to navigation Jump to search
At about 17:00 UTC Feb. 22 the www.wikipedia.org page was severely broken for about an hour.
- A bug was filed at around 17:09 UTC Feb.22 noting that the text on www.wikipedia.org is invisible. task T158782
- We were made aware of this bug at about 17:40 UTC
- at 18:15 UTC an attempt was made to rollback to the previous deploy. The deploy was visible on mwdebug1002 without error, but the error persisted in production.
- The wikipedia.org portal depends on a specific order of syncing followed by purging urls, which is fragile and needs some rethinking.