Incident status

• Unintentional +2 on a config patch without deployment

• eqiad/LVS

• wdqs CODFW 5xx errors

No incidents reported in the last 14 days.

Create a new incident report

1. Deal with the issue by following Incident response/Runbook.
2. Use this form to create a report based on the report template:
   
3. Write up what happened, filling out the template. Use past incident reports for inspiration.

Past incidents

This page tracks incident reports about Wikimedia Foundation properties, such as Wikipedia.

2023

• 2023-01-09 API errors - db1143
• 2023-01-10 eqsin network outage
• 2023-01-14 asw-b2-codfw failure
• 2023-01-17 MediaWiki
• 2023-01-17 asw-b2-codfw failure redux
• 2023-01-24 sessionstore quorum issues
• 2023-01-30 kartotherian
• 2023-02-07 mediawiki.page-undelete event stream
• 2023-02-11 logstash latency
• 2023-02-22 read only
• 2023-02-22 wiki outage
• 2023-02-23 PHP worker threads exhaustion
• 2023-02-28 GitLab data loss
• 2023-03-09 mailman
• 2023-04-17 eqiad/LVS
• 2023-05-02 Phabricator outage during GitLab maintenance
• 2023-05-05 prometheus down in ulsfo and eqsin
• 2023-05-05 wdqs not updating in codfw
• 2023-05-19 videoscaler/jobrunner
• 2023-05-23 wdqs CODFW 5xx errors
• 2023-05-25 eqiad/LVS
• 2023-05-30 Unintentional +2 on a config patch without deployment

Incident history

Check Category:Incident documentation for older incident reports.

See also

• https://www.wikimediastatus.net (docs).

• Incident graphs by Krinkle.

• #wikimedia-tech ^connect on Libera Chat