Tool:Early warning bot
 Early warning bot
| |
|---|---|
| Website | https://earlywarningbot.toolforge.org/ |
| Description | Provide feedback on a patch as soon as any of its jobs fail. |
| Keywords | quibble, bot, ci |
| Author(s) | Kosta Harlantalk |
| Maintainer(s) | Kosta Harlan (View all) |
| Source code | https://gitlab.wikimedia.org/kharlan/earlywarningbot |
| License | MIT License |
What
Early warning bot comments and/or votes on a Gerrit change as soon as any of the jobs for the change fail.
Why
For MediaWiki core and extensions, some jobs (e.g. Selenium) can take over 15 minutes to complete. Meanwhile, static analysis jobs like phan, phpcs, eslint complete within minutes. It is nice to know as soon as job fails, so that you can make a fix without needing to wait ~20 minutes for all jobs to complete. Yes, you can watch Zuul, but receiving a notification via email/IRC can be nice.
How
Quibble 1.5.0 accepts a --reporting-url argument. In Wikimedia CI, this is configured to point to https://earlywarningbot.toolforge.org/. When a job fails, Quibble checks to see if there repository that triggered the jobs has a quibble.yaml file (documentation). If the file exists, Quibble POSTs information to the endpoint with the job name, a URL to the build failure, and whether the repository is configured to receive comments and/or votes (Verified -1) on build failure.
The bot is deployed to earlywarningbot.toolforge.org and uses an API key to validate that the request is from Quibble, it then posts on the Gerrit change. If should_comment: 1 is in the project's quibble.yaml, the bot will leave a comment on the patch indicating that a job failed, with a link to the build URL. If should_vote: 1 is set in the project's quibble.yaml, the bot will vote Verified: -1 on the patch. (This is what jenkins-bot will eventually do when all jobs are finished for that patch.)
Using with your project
Create a quibble.yaml file with the following contents and commit it to your repository.
earlywarning:
should_vote: 1
should_comment: 1
To switch off the integration with the bot, change the values to 0 or delete the file.
Known issues
- If a build fails because of a flaky test, and the bot is configured to vote, then running "recheck" would still leave a
Verified: -1on the patch. This may or may not then interfere with a +2 on the patch. The workaround is to manually remove theVerified: -1vote after submitting arecheckcomment.
Deployment
- Merge the merge request
- SSH to toolforge
become earlywarningbot- Run
./build-and-release.sh
#!/bin/sh
set -ex
cd earlywarningbot
git pull
jsub -N build -mem 2G -sync y -cwd cargo build --release
cd ~/
webservice restart
We have a webservice template:
backend: kubernetes
type: golang1.11
extra_args:
- /data/project/earlywarningbot/start.sh
And start.sh contains credentials used for the bot:
#!/bin/sh
ROCKET_ADDRESS=0.0.0.0 ROCKET_LOG_LEVEL=normal ROCKET_KEY={redacted} ROCKET_GERRIT_URL="https://gerrit.wikimedia.org" ROCKET_GERRIT_USERNAME="earlywarningbot" ROCKET_GERRIT_PASSWORD="{redacted}" ./earlywarningbot/target/release/earlywarning
Monitoring
kubectl get podskubectl logs -f earlywarningbot-59476d45c9-th9j5