Incidents/2018-12-18 train-1.33.0-wmf.9

From Wikitech

Summary

There were several problems with the 2018-12-18 1.33.0-wmf.9 train (phab:T206663) resulting in 0 rollbacks.

Stats

Issues

  • phab:T211885 ErrorException from line 47 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Kartographer/includes/ApiQueryMapData.php: PHP Warning: data error
  • phab:T212217 ErrorException from line 317 of /srv/mediawiki/php-1.33.0-wmf.9/extensions/ExtensionDistributor/includes/specials/SpecialBaseDistributor.php: PHP Notice: Undefined variable: downloadImg
  • phab:T212218 Fatal error: Call to undefined function Kartographer\Wikimedia\suppressWarnings() in /srv/mediawiki/php-1.33.0-wmf.9/extensions/Kartographer/includes/ApiQueryMapData.php on line 49
  • phab:T212427 No namespace configured for entity type `form`

Timeline

This is a step by step outline of what happened to cause the incident and how it was remedied.

2018-12-13

  • 14:46 phab:T211885 ErrorException from line 47 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Kartographer/includes/ApiQueryMapData.php: PHP Warning: data error

2018-12-18

  • 14:40 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: group0 to 1.33.0-wmf.9
  • 14:47 phab:T212217 ErrorException from line 317 of /srv/mediawiki/php-1.33.0-wmf.9/extensions/ExtensionDistributor/includes/specials/SpecialBaseDistributor.php: PHP Notice: Undefined variable: downloadImg
  • 15:01 phab:T212218 Fatal error: Call to undefined function Kartographer\Wikimedia\suppressWarnings() in /srv/mediawiki/php-1.33.0-wmf.9/extensions/Kartographer/includes/ApiQueryMapData.php on line 49

End of Day Summary

  • Group0: 1.33.0-wmf.9
  • Group1: 1.33.0-wmf.8
  • Group2: 1.33.0-wmf.8
  • Status: Yes Done

2018-12-19

  • 12:33 * phab:T212218 Fatal error: Call to undefined function Kartographer\Wikimedia\suppressWarnings() in /srv/mediawiki/php-1.33.0-wmf.9/extensions/Kartographer/includes/ApiQueryMapData.php on line 49
  • 12:43 phab:T211885 ErrorException from line 47 of /srv/mediawiki/php-1.33.0-wmf.8/extensions/Kartographer/includes/ApiQueryMapData.php: PHP Warning: data error
  • 16:59 phab:T212217 ErrorException from line 317 of /srv/mediawiki/php-1.33.0-wmf.9/extensions/ExtensionDistributor/includes/specials/SpecialBaseDistributor.php: PHP Notice: Undefined variable: downloadImg
  • 20:03 <hashar@deploy1001> Synchronized php: group1 wikis to 1.33.0-wmf.9 (duration: 00m 51s)

End of Day Summary

  • Group0: 1.33.0-wmf.9
  • Group1: 1.33.0-wmf.9
  • Group2: 1.33.0-wmf.8
  • Status: Yes Done

2018-12-20

  • 14:28 phab:T212427 No namespace configured for entity type `form`
  • 16:59 <zfilipin@deploy1001> rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.9
  • 17:02 phab:T212427 No namespace configured for entity type `form`

End of Day Summary

  • Group0: 1.33.0-wmf.9
  • Group1: 1.33.0-wmf.9
  • Group2: 1.33.0-wmf.9
  • Status: Yes Done

Conclusions

What weakness did we learn about and how can we address them?

Links to relevant documentation

Actionables

Explicit next steps to prevent this from happening again as much as possible, with Phabricator tasks linked for every step.

NOTE: Please add the #wikimedia-incident Phabricator project to these follow-up tasks and move them to the "follow-up/actionable" column.

  • To do #1 (TODO: Create task)
  • To do #2 (TODO: Create task)
  • …
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Incidents/2018-12-18_train-1.33.0-wmf.9&oldid=1966169"