User:Beetstra/MiniManual

From Wikitech

Short manual to start the bots.

Bots

At the moment, I have 4 bots running from Labs:

  • linkwatcher - on bots-liwa
  • coibot - on bots-nr1
  • xlinkbot - on bots-nr1
  • unblockbot - on bots-nr1

The bots have a main procedure, and run a good handful of modules. To see if the bot itself is running":

  • linkwatcher: 'ps -Af | grep linkwatcher.pl'
  • coibot: 'ps -Af | grep coibot.pl'
  • xlinkbot: 'ps -Af | grep xlinkbot.pl'
  • unblockbot: 'ps -Af | grep unblockbot.pl'

Killing the bots

If they are running, and are online, and you need to restart them, go to a channel where they are (#BeetstraBotChannel is the common channel for all bots), and type to kill the bots (see notes on modules below first!):

  • linkwatcher: 'LiWa3_1 !quit' (see note on modules below)
  • coibot: 'COIBot quit' (see note on modules below)
  • xlinkbot: 'XLinkBot quit' (see note on modules below)
  • unblockbot: 'UnBlockBot quit'

If the bots don't listen (to you), or if the bots are not there but still running, kill the main procedure

  • the 'ps' command above should have given you a PID, type 'kill <PID>'
  • Check whether the modules for that bot are running (sometimes they don't die) - see #Modules

Starting the bots

All the bots (except unblockbot) have a different number of modules running, they should now die automatically (which may take some time). Wait 30 secs or so, and have a look:

  • linkwatcher: 'ps -Af | grep LiWa3'
  • coibot: 'ps -Af | grep COIBot'
  • xlinkbot: 'ps -Af | grep XLinkBot'

Make sure that all of these modules for a given bot are gone before restarting the bot. To start the bots:

  • go to the right instance to run the bot (bots-nr1 for COIBot, UnBlockBot and XLinkBot, bots-liwa for LiWa3), and go into the right subdirectory
  • run the bot:
    • linkwatcher:
      • Login to bots-liwa
      • Go to /mnt/share/linkwatcher
      • run 'nohup perl linkwatcher.pl &'
    • coibot:
      • Login to bots-nr1
      • Go to /mnt/share/coibot
      • run 'nohup perl coibot.pl &'
    • xlinkbot: 'nohup perl xlinkbot.pl &'
      • Login to bots-nr1
      • Go to /mnt/share/xlinkbot
      • run 'nohup perl xlinkbot.pl &'
    • unblockbot:
      • Login to bots-nr1
      • Go to /mnt/share/unblockbot
      • run 'nohup perl unblockbot.pl &'

The bots should now start, and automatically start their modules in appropriate predetermined numbers set in the settings

Modules

If parts of the bot are not working, it may be due to one module being dead, hanging or fatally crashed. In principle, when a module crashes, the bot should re-start it automatically. Error messages should be in the nohup.out file

Some of the bots have built-in killing and restarting features for the modules - these procedures are contained in the main bot. Otherwise it is possible to do 'nohup perl <modulename> &' to start them from here.

In general it is save to just kill a hanging module and let the main bot respawn them. If they do not respawn, it may be that the main bot is hanging for some strange reason. I have noticed cases where modules are hanging. I have not figured out why (if you see a reason, please tell me, I might be able to work around it). It is often detectable if a module is hanging by checking the sequence numbers. It is after weeks of running generally a good plan to kill all modules and force them to all start fresh.


  • linkwatcher ('ps -Af | grep LiWa3'):
    • DiffReader.pl: ~8, each parsing about 100 channels on irc.wikimedia.org, which channels they parse is written into ~8 files in the root directory of the bot
    • LinkParser.pl: multiple, these guys do the actual parsing of each diff, you need enough to keep up (command '!info' on freenode, 'P'-queues should be 0 or close to 0)
    • LinkAnalyser.pl: multiple, these guys store all the external links found on bots-sql2, and do the statistics on them ('A'-queues)
    • LinkReporter.pl: 3, these guys connect to freenode as LiWa3_1, LiWa3_2 and LiWa3_3. 3 seem to be fine at the moment, no flooding of freenode.
  • coibot ('ps -Af | grep COIBot');
    • DiffReader.pl: 1 (connects to all channels on irc.wikimedia.org, I may need to change this in the future to the same system as linkwatcher)
    • Parser.pl: multiple, these do the COI parsing
    • Case.pl: 1 - case handling. Have ONLY ONE running, otherwise commands WILL be confusing on IRC
    • Commander.pl: 1 - handles practically ALL commands given on Freenode to COIBot. Keep only 1 running to avoid confusion, though there is not much data-sharing.
  • xlinkbot ('ps -Af | grep XLinkBot'):
    • DiffReader.pl: 1 (connects to #en.wikipedia on irc.wikimedia.org)
    • Reverter.pl: multiple (these do the actual reverting, you need a couple so they can revert simultanuously, you don't need many)
  • unblockbot
    • No modules

Modules carry the name of their starting bot as a parameter and a sequence number (incremental number, will increase if you kill the module, will give a sequential set of numbers if there are more than one module running). Some of the modules also carry other parameters for internal operation.

Passwords

The bots that are editing (coibot, xlinkbot) have a cookie file with their password. If they are running while logged out, do the following:

  • coibot: 'perl coibot.pl password=<password>'
  • xlinkbot: 'perl xlinkbot.pl password=<password>'

They will comment whether login is successful or not and die if there is a password in the parameter (otherwise the password would be visible in a 'ps'). Modules that edit (LinkSaver.pl, Reverter.pl, Case.pl, Commander.pl) should pick up that the editing is working (most do a login-check before every edit), otherwise kill them to get them to edit again.

If the bots are not running, you may first have to force it to login using these commands, and then run them as normally (the bots will not start when they detect they are not logged in).

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=User:Beetstra/MiniManual&oldid=9917"