Tool:Stashbot

From Wikitech
Jump to: navigation, search
Tool Labs tools
Crystal Clear app package utilities.png
Website https://tools.wmflabs.org/stashbot/
Description An IRC bot designed to store data in an Elasticsearch cluster
Keywords irc, python
Author(s) BryanDavis
Maintainer(s) BryanDavis, Yuvipanda, Greg Grossmeier (View all)
Source code gerrit (diffusion)
License GNU General Public License 3.0 or later
Issues Open tasks · Report a bug
Admin log Tools.stashbot/SAL

Stashbot is an IRC bot that stores data for use by other tools, responds to !log messages, and gives information about Phabricator objects referenced in messages.

Data collected by stashbot is stored in an Elasticsearch cluster in Tool Labs and consumed by:

  • Bash
  • SAL
  • (an as yet unwritten IRC history search system)

Interacting with the bot

The bot archives all messages in IRC channels it joins to Elasticsearch indices that rotate monthly. It also examines the messages to see if they contain commands or other triggers.

!log processing

In the #wikimedia-operations connect, #wikimedia-cloud connect, #wikimedia-releng connect, #wikimedia-analytics connect, and #countervandalism connect channels, messages beginning with !log are stored in a special index that is used by the SAL tool. The messages are also written to wiki pages on either wikitech or mediawiki.org. In the #wikimedia-operations channel, the messages are posted as tweets to the @wikimediatech Twitter account.

Phabricator tasks mentioned in !log messages will also be updated to include a reference to the SAL entry. This is commonly used in SWAT deployments and when performing other system maintenance related to an open issue.

!log messages in the #wikimedia-cloud channel are expected to begin with the name of a Labs project or Tool Labs account. For example: !log tools.stashbot Restarting to use updated configuration. The project/tool name is removed from the message and used to determine the storage location in both Elasticsearch and on-wiki. The bot will validate this project prefix using data from LDAP and tell you if the project is unknown.

The processing of !log messages in a channel can be limited to certain users by configuring an access control list of allowed or disallowed accounts. This feature can be used temporarily or permanently to block abuse by vandals.

!bash processing

Messages in an channel that begin with !bash are added to a special Elasticsearch index that is used by the Bash tool. See the tool's help page for more information.

Phabricator mentions

Differential, Pholio, and Maniphest objects mentioned by id in irc messages (e.g. T1234) will be looked up Phabricator and then the title and url of the object will be reported back to the channel. This feature can be disabled on a per-channel basis if the channel users decide that it is disruptive.

Maintenance

The bot runs from Tool Labs under the "stashbot" account as a kubernetes pod.

# Start the bot
$ ./bin/stashbot.sh start

# Stop the bot
$ ./bin/stashbot.sh stop

# Restart the bot
$ ./bin/stashbot.sh restart

# Check job status
$ ./bin/stashbot.sh status

# Tail log files
$ ./bin/stashbot.sh tail

# Update bot to latest version
$ ./bin/stashbot.sh udpdate
$ ./bin/stashbot.sh restart

Project history

Stashbot started as a replacement for a set of Logstash filters deployed in the the beta cluster Logstash instance. The initial Logstash based system was functional, but had many limitations on the amount of processing that could be done. There was also no way for the irc connection made by Logstash to be used to send information back to the channels.

The code for the bot has borrowed heavily from Morebots and Jouncebot. Some innovations from stashbot have also been "upstreamed" back to jouncebot.

See Also