Incident documentation

No incidents reported in the last 8 days.

Create a new incident report

1. Deal with the issue (See also Incident response).
2. Decide whether to write a lightweight or full-length report.
3. Create a page for your report using one of the forms below.
   Lightweight report (template):
   
   Full-length report (template):
4. Write up what happened, filling out the template. Use past incident reports for inspiration.
5. Once done, change the status to "in-review", preferably within 7 days of the incident.
   Within 3 weeks the ONFIRE working group will get back to you with feedback.
   After the ONFIRE review and any updates to the report, set the status to "final" and notify the ops@ mailing list.

Past incidents

This page tracks incident reports about Wikimedia Foundation properties, such as Wikipedia.

2020

• 20200108-mw-api
• 20200126-app server latency
• 20200127-app server latency
• 20200204-app server latency
• 20200204-maps
• 20200206-mediawiki
• 20200207-mediawiki API down
• 20200207-wikidata
• 20200211-caching-proxies
• 20200224-esams
• 20200225-mediawiki interface language
• 20200319-parsercache
• 20200325-codfw-network
• 20200407-Wikidata's wb items per site table dropped
• 20200501-vc-link-failure
• 20200505-wdqs-deploy
• 20200511-thumbor
• 20200522-thumbnails
• 20200528-commons read-only
• 20200604-cloud-private-repo
• 20200611-sessionstore+kubernetes
• 20200619-Logstash
• 20200625-caching-sessions
• 20200703-Lilypond down
• 20200705-eqsin-router-crash
• 20200714-termbox and wikifeeds timeouts
• 20200723-wdqs-outage
• 20200812-appservers oom
• 20200814-isp-unreachable
• 20200901-data-center-switchover
• 20200902-wdqs-outage
• 20200908-Rack D hosts outage
• 20200909-mobileapps config change
• 20200925-s5-replication-lag

Incident history

Check Category:Incident documentation for older incident reports.