Tool:DrTrigonBot/HowTo and Information
- mw:Wikimedia Labs/Tool Labs/Needed Toolserver features
- Nova Resource:Tools/Help
- Setting up Flask cgi app as a tool - seams to be the preferred python web framework on tool-labs (e.g. used by valhallasw, Sumana, me and more)
- interesting and useful pages from Toolserver Wiki
$ ssh email@example.com drtrigon@tools-login:~$ become drtrigonbot local-drtrigonbot@tools-login:~$
We want e.g. to copy directory ~/source to tools-login.wmflabs.org:/data/project/drtrigonbot/dest under the ownership of local-drtrigonbot.
Login and create temporary directories:
$ ssh firstname.lastname@example.org drtrigon@tools-login:~$ mkdir temp drtrigon@tools-login:~$ cd temp drtrigon@tools-login:~$ mkdir dest drtrigon@tools-login:~$ exit
Copy the files to the server (by using scp):
$ scp -r ~/source/* email@example.com:~/temp/dest
this might take some while. Attention this might miss .files may be you have to use
$ scp -r ~/source firstname.lastname@example.org:~/temp
Login again and move them to the project:
$ ssh email@example.com drtrigon@tools-login:~$ cp -r ~/temp /data/project/drtrigonbot drtrigon@tools-login:~$ ls -la /data/project/drtrigonbot
as you can see the files are there but under the wrong ownership. Some of them might have given read and even write permission for the group as well, but just to make sure enter:
drtrigon@tools-login:~$ chmod -R g+rw /data/project/drtrigonbot/temp
from here you can also start using e.g. mc (midnight commander).
Adopt ownership and finalize:
drtrigon@tools-login:~$ become drtrigonbot local-drtrigonbot@tools-login:~$ cp -r ~/temp/dest ~
copying changes also the ownership, and everything is ok now. Important in this step is to copy and not move the files in order to create new files under correct ownership.
local-drtrigonbot@tools-login:~$ rm -r ~/temp local-drtrigonbot@tools-login:~$ exit drtrigon@tools-login:~$ rm -r ~/temp
or use mc for clean-up, which is less risky!
Multiple steps are listed here, go through them starting from the first and continue until the bug is solved.
- check what error occurred and where, please use the panel (see status under DrTrigonBot#Information) for this purpose (confer #Status explanation) or the web tools (debug) output
- check SGE online (see server under DrTrigonBot#Information) and if this does not help by #Login and using
qstatas well as
qacctaccording to Nova Resource:Tools/Help#Submitting, managing and scheduling jobs on the grid
proposal for new one (not used at the moment) that should make debugging (especially for 'n/a' states) easier and thus faster (debugging is the main purpose):
|usual operational states|
|ok||everything ok, bot operates as expected (not running at the moment)||
|run||the bot is running at the moment (default for continuous running bots)||
|issue and error states|
|n/a||the bot is in an unknown state that might need attention (e.g. crashed silently)||
|err||the bot has finished with an error (but should start again next time)||
|off and stand-by states|
|off||bot down resp. switched off (no run nor started since some time)||
Alternatives for 'n/a': , / commons:Category:Dots/Pog-Set / May be swap 'n/a' and 'err' colors... what is more severe? What is the time order during run?
Needs to change some of the interpreting (e.g. subster) codes because of image changes: off → err. Even worse all image links have changed to the .svg.png variant.
bots architecture [partly obsolete]
Using Agent Forwarding
$ eval `ssh-agent` $ ssh-add ~/.ssh/your_key_file_for_labs
$ ssh -A firstname.lastname@example.org #once in bastion $ ssh bots-4 #or $ ssh bots-4.pmtpa.wmflabs
Logs / Logging
- Nova Resource:Bots#Server Admin Log
Install software packages in root/sudo environment (e.g. bots-4) from shell:
sudo apt-get install ...
Install software packages in non-root/sudo (e.g. bots-apache01) through gerrit, check bug 53704.
(git clone ...) (adopt according sub-bugs of bug 53704) (push for review) (notify somebody and wait)