Incidents/20130617-AccountCreation
Summary
Due to code in VisualEditor's A/B testing sorting on new accounts, newly created user accounts were not having their email and password set. Thus, many thousands of new accounts were only partially created.
Reported as bugzilla:49727.
What Happened
Basically, this commit 68845 caused the whole thing. This one 69277 fixed it.
Lessons Learned
Our logging works, but we don't monitor those logs and report spikes/errors.
Action Items
- Errors and exceptions are currently broadcast to fluorine and vanadium via UDP. Ori has code that parses the stream and generates the Ganglia graphs, but it isn't hooked up to Icinga or any other form of monitoring. We need someone from Ops to pair with Ori to set this up. This is bugzilla:49757.
Done
