Help:Toolforge/Running Pywikibot scripts

Toolforge

• Cloud Services overview
• Toolforge user docs
• Toolforge changelog

Get started

• Quickstart: set up and get access
• How Toolforge works
• Rules you must follow
• Tutorials

Build and run tools

• Navigate tool accounts and files
• Build container images for tools
• Run a web service
• Schedule and manage jobs
• Manage tool runtime configuration (envvars)
• Language-specific details:
  • Python
  • Pywikibot
  • Node.js
  • PHP
  • ...more languages/frameworks
• Use Redis for caching
• Index content with Elasticsearch

Access shared storage and databases

• Access shared storage and public wiki dumps
• Access the Wiki Replicas databases
• Access replica search indices
• Manage tool databases

Share and maintain tools

• Set up version control and code review
• Develop successful tools
• Find and share tools on Toolhub
• Delete a tool
• Contribute to Toolforge

Get help

• How and where to get help
• Troubleshooting

Useful links

• Toolforge admin docs
• List of tools
• Toolforge Admin Console (toolsadmin)

[]

The Pywikibot framework is a Python library and collection of scripts to automate work on MediaWiki sites. This tutorial contains instructions on how to run Pywikibot's built-in scripts on Toolforge using the Toolforge jobs framework.

If you want to run a script that is not included with Pywikibot itself, follow the advanced Pywikibot tutorial.

Prerequisites

To effectively use Pywikibot on Toolforge, you need:

• Access to a tool account on Toolforge. See Help:Toolforge/Quickstart to learn how to set it up and use. Note that to use Toolforge you need a beginner-level understanding of Linux terminal, SSH (see Internet Technologies/SSH), and Bash (see Bash Shell Scripting).
• Basic familiarity with Python and Pywikibot. You don't need to know how to program in Python but it helps to understand how to run Python scripts. To learn Python, see Non-Programmer's Tutorial for Python 3 or The Python Tutorial in Python documentation.For information about Pywikibot, see Manual:Pywikibot.

Setup

The provided Pywikibot image uses OAuth for authenticating with Wikimedia wikis. To set it up:

1. Set up an owner-only OAuth 1.0a credential following the instructions at mw:Manual:Pywikibot/OAuth/Wikimedia#Registering your bot with the wiki software. The required dependencies are already installed and the configuration is included in the image.
2. On Toolforge, create environment variables that contain the bot username and the four values you got when creating the OAuth credential:

   tools.mytool@tools-sgebastion-10:~$ toolforge envvars create PWB_USERNAME "Example"
   tools.mytool@tools-sgebastion-10:~$ toolforge envvars create PWB_CONSUMER_TOKEN "..."
   tools.mytool@tools-sgebastion-10:~$ toolforge envvars create PWB_CONSUMER_SECRET "..."
   tools.mytool@tools-sgebastion-10:~$ toolforge envvars create PWB_ACCESS_TOKEN "..."
   tools.mytool@tools-sgebastion-10:~$ toolforge envvars create PWB_ACCESS_SECRET "..."
   

3. You are now ready to run Pywikibot scripts on Toolforge.

Running scripts

Use the toolforge jobs utility to run jobs using the jobs framework. The tool-pywikibot/pywikibot-scripts-stable:latest image is maintained by the Toolforge admin team and always points to the latest stable Pywikibot release.

This image and its method of running scripts does not read or write files in the tool's $HOME directory. This includes files like user-config.py and user-fixes.py that you may be used to using with other Pywikibot deployments.

Examples

To run a single job:

tools.mytool@tools-sgebastion-10:~$ toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -family:PROJECT -lang:LANGUAGE SCRIPT_NAME arg1 -always" JOB_NAME

for example, to run the redirect.py script on Wikitech, you would use:

tools.mytool@tools-sgebastion-10:~$ toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -family:wikitech -lang:en redirect double -always" fix-double-redirects

You can use the --schedule option to run a job on a timer, for example every day:

tools.mytool@tools-sgebastion-10:~$ toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb -family:PROJECT -lang:LANGUAGE SCRIPT_NAME arg1 -always" --schedule "@daily" JOB_NAME

See also

• Help:Toolforge/Running Pywikibot scripts (advanced)
• Toolforge admin documentation for maintaining the image

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support

• Chat in real time in the IRC channel #wikimedia-cloud ^connect or the bridged Telegram group
• Discuss via email after you have subscribed to the cloud@ mailing list

Stay aware of critical changes and plans

• Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
• Read the News wiki page

Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)