Tool:Stashbot
| |
|---|---|
| Website | https://stashbot.toolforge.org/ |
| Description | An IRC bot for storing data in Elasticsearch |
| Keywords | irc, python |
| Author(s) | BryanDavis |
| Maintainer(s) | BryanDavis, Yuvipanda, Greg Grossmeier (View all) |
| Source code | gerrit (diffusion) |
| License | GNU General Public License 3.0 or later |
| Issues | Open tasks · Report a bug |
| Admin log | Tools.stashbot/SAL |
Stashbot is an IRC bot that stores data for use by other tools, responds to !log messages, and gives information about Phabricator objects referenced in messages.
Data collected by stashbot is stored in an Elasticsearch cluster in Toolforge and consumed by:
Interacting with the bot
The bot archives all messages in IRC channels it joins to Elasticsearch indices that rotate monthly. It also examines the messages to see if they contain commands or other triggers.
!log processing
In the #wikimedia-operations connect, #wikimedia-sre connect, #wikimedia-fundraising connect, #wikimedia-traffic connect,#wikimedia-cloud connect, #wikimedia-releng connect, #wikimedia-analytics connect, and #countervandalism connect channels, messages beginning with !log are stored in a special index that is used by the SAL tool. The messages are also written to wiki pages on either wikitech or mediawiki.org. In the #wikimedia-operations channel, the messages are posted to the @wikimedia_sal Mastodon account.
Phabricator tasks mentioned in !log messages will also be updated to include a reference to the SAL entry. This is commonly used in backport windows and when performing other system maintenance related to an open issue.
!log messages in the #wikimedia-cloud connect channel are expected to begin with the name of a Cloud VPS project or Toolforge tool account (prefixed with tools.). For example: !log tools.stashbot Restarting to use updated configuration. The project/tool name is removed from the message and used to determine the storage location in both Elasticsearch and on-wiki. The bot will validate this project prefix using data from LDAP and tell you if the project is unknown.
The processing of !log messages in a channel can be limited to certain users by configuring an access control list of allowed or disallowed accounts. This feature can be used temporarily or permanently to block abuse by vandals.
!bash processing
Messages in a channel that begin with !bash are added to a special Elasticsearch index that is used by the Bash tool. See the tool's help page for more information.
Phabricator mentions
Differential, Pholio, and Maniphest objects mentioned by id in irc messages (e.g. T1234) will be looked up in Phabricator and then the title and URL of the object will be reported back to the channel. This feature can be disabled on a per-channel basis if the channel users decide that it is disruptive.
Maintenance
The bot runs from Toolforge under the "stashbot" account as a kubernetes pod.
# Start the bot
$ ./bin/stashbot.sh start
# Stop the bot
$ ./bin/stashbot.sh stop
# Restart the bot
$ ./bin/stashbot.sh restart
# Check job status
$ ./bin/stashbot.sh status
# Tail log files
$ ./bin/stashbot.sh tail
# Update bot to latest version
$ ./bin/stashbot.sh update
$ ./bin/stashbot.sh restart
Joining a new channel
Maintainers/Members/Admins of the tool can do the below to enable the bot to join a new channel;
$ ssh login.toolforge.org
$ become stashbot
$ vim etc/config-libera.yaml
:# Add the channel in the irc.channels list.
:# The list is in alphabetical order in 2 sections (priority channels, other channels)
$ bin/stashbot.sh restart
Project history
Stashbot started as a replacement for a set of Logstash filters deployed in the the beta cluster Logstash instance. The initial Logstash based system was functional, but had many limitations on the amount of processing that could be done. There was also no way for the irc connection made by Logstash to be used to send information back to the channels.
The code for the bot has borrowed heavily from Morebots and Jouncebot. Some innovations from stashbot have also been "upstreamed" back to jouncebot as part of the ib3 python library.