Incident status

No incidents reported in the last 14 days.

Create a new incident report

1. Deal with the issue (See also Incident response).
2. Decide whether to write a lightweight or full-length report, and use the below form to start.
   Lightweight report (template):
   
   Full-length report (template):
3. Write up what happened, filling out the template. Use past incident reports for inspiration.
4. Once done, change the status to "in-review", preferably within 7 days of the incident.
   Within 3 weeks the ONFIRE working group will get back to you with feedback.
   After the ONFIRE review and any updates to the report, set the status to "final" and notify the ops@ mailing list.

Past incidents

This page tracks incident reports about Wikimedia Foundation properties, such as Wikipedia.

2021

• 2021-01-16 appserver latency
• 2021-02-01 swift-codfw
• 2021-02-05 wmflabs certs expired
• 2021-02-26 sudo
• 2021-03-14 MediaWiki API
• 2021-03-30 Jobqueue overload
• 2021-04-03 mailman2 attachments
• 2021-04-06 partial rack codfw
• 2021-04-15 appserver latency
• 2021-04-20 MediaWiki API slowdown
• 2021-04-26 thumbor
• 2021-04-27 Commons wiki primary db
• 2021-04-29 db and memc load
• 2021-06-15 Eqsin network
• 2021-06-29 trwikivoyage primary db
• 2021-06-30 Ulsfo traffic surge
• 2021-07-14 eventgate-analytics latency spike caused MW app server overload
• 2021-07-16 asw-a2-codfw network
• 2021-07-26 ruwikinews DynamicPageList
• 2021-09-01 partial parsoid outage
• 2021-09-04 appserver latency
• 2021-09-06 Wikifeeds
• 2021-09-12 Esams upload
• 2021-09-13 cirrussearch restart
• 2021-09-18 appserver latency
• 2021-09-26 appserver latency
• 2021-09-29 eqiad-kubernetes
• 2021-10-08 network provider
• 2021-10-22 eqiad return path timeouts
• 2021-10-25 s3 db recentchanges replica
• 2021-10-29 graphite
• 2021-11-02 Cloud VPS networking
• 2021-11-04 large file upload timeouts
• 2021-11-05 TOC language converter
• 2021-11-10 cirrussearch commonsfile outage
• 2021-11-18 codfw ipv6 network
• 2021-11-23 Core Network Routing
• 2021-11-25 eventgate-main outage
• 2021-12-03 mx

Incident history

Check Category:Incident documentation for older incident reports.