Help:Toolforge/Tool accounts

This page explains what a tool account is, how to create a tool account/tool, and how to add and remove maintainers. For one-time setup steps to get started with Toolforge, see the Toolforge quickstart.

Key concepts

Tools and tool accounts

The terms "tool", "tool account", and "project" have the same meaning in Toolforge; "tool accounts" and "tool" are often used interchangeably. The tool is the basic unit of deployment in Toolforge. Each tool is actually a tool account with resources, processes, and other components in a tool-specific namespace.

A tool account is a group account associated with a tool. A tool account can have one or more members or tool maintainers. You create a separate tool account for each new tool you develop on Toolforge. When you're invited to work on or help maintain a tool, you'll join an existing tool account. Tool accounts enable multiple maintainers to collaboratively manage the software source code, configuration, and jobs for that tool.

Each tool account includes:

A home directory on shared storage: /data/project/<TOOL NAME>
The ability to run a Web service which is visible at https://<TOOL NAME>.toolforge.org/
Database access credentials: $HOME/replica.my.cnf, which provide access to the production database replicas as well as to project-local databases
Access to the job and task queues
Credentials and a namespace for running containers on the Kubernetes cluster

Maintainers

People who have access to a tool account are called maintainers. Maintainers have access to the tool account's code and data.

Maintainers can:

Create tool accounts/tools
Join existing tool accounts/tools
Leave tool accounts/tools in the care of others
Log in (sudo) to the tool accounts/tools

Join an existing tool account

All tool accounts hosted in Toolforge are listed on the tools list. Contact the maintainer to ask them if you can join an existing tool account. The maintainer can follow the instructions below to add you to that tool account.

Add or remove maintainers

Existing maintainers can follow these instructions to add new maintainers to a tool account:

Log in to the Toolforge admin console.
Navigate to the tool listing and select the tool you want to manage from the left sidebar.
Click "Manage maintainers". Add the new maintainers and click "Update".

To add a user as a maintainer, that user must already have applied to join Toolforge or their name won't appear in the autocompletion list.

Create a new tool account / tool

To create a new tool, create a new tool account in toolsadmin:

Go to https://toolsadmin.wikimedia.org/tools/.
Click on the "Tools" tab.
Click the "Create new tool" link at the bottom of the "Your tools" sidebar.
Follow the instructions in the tool account creation form. Note the instructions below about Naming your tool.
After you create the tool account, log off, then log back in to access the new tool account.

Note: If you only recently received access to the tools project, you may get an error about appropriate credentials. Log out and back in to fix the issue.

If you were logged in through ssh when you created the tool account, you must log off and log in again.

Name a tool account / tool

The tool account and tool have the same name. This name is included in the URL for the web service, if the tool has one. Make sure the name is appropriate and is spelled correctly.

Rename a tool account / tool

Tools can't be renamed. You can create a new tool with a new name and copy the code over from the old tool.

Switch to / become a tool account

Whenever you log in to Toolforge, you first SSH to the bastion host, but then you must "become" the tool account you want to work on. Become your tool by using the become command:

$ become <toolname>

You should see the command prompt change to:

tools.<toolname>@tools-bastion:~$

Troubleshooting: no such tool

$ become <TOOL NAME>
become: no such tool '<TOOL NAME>'

Wait a few minutes for the tool account creation to complete.
Check that the spelling of the tool name is correct.

Troubleshooting: You are not a member of the group tools

$ become <TOOL NAME>
You are not a member of the group tools.<TOOL NAME>.
Any existing member of the tool's group can add you to that.

If you are already logged in via SSH when you create a new tool, log out, and log in again to activate your new permissions.

Manage files in Toolforge

Tool accounts store code and data in shared storage. Tool maintainers may need to:

manage file permissions;
transfer files to Toolforge;
take ownership of transferred files;
synchronize files with rsync;
work with files locally using sshfs;
share files and code between tools.

Detailed instructions, examples, best practices, and troubleshooting information are available at Managing and sharing files in Toolforge.

Scripting

If you want to script the SSH access to your tool, you can concatenate your scripted command after running become, for example:

user@laptop:~ $ ssh dev.toolforge.org become mytool "bash -c 'echo hello \$LOGNAME \$PWD'"
hello tools.mytool /mnt/nfs/labstore-secondary-tools-project/mytool

The command inside the bash -c call will get executed inside the become context.

Note shell escaping may be needed so variables are not evaluated in the outer shell, but in the inner one, like in the example.

Add a description to your tool account / tool

Each tool can provide a description (example) by creating a toolinfo record using https://toolsadmin.wikimedia.org/tools/. Navigate to your tool's record in ToolsAdmin and look for the "Add toolinfo" button. See the Toolhub documentation for other ways to add a toolinfo record, and details about how to populate the fields.

Delete a tool account

Mark a tool for deletion by using the "Disable tool" button on the tool's detail page on https://toolsadmin.wikimedia.org/. Disabling a tool will immediately stop any running jobs including webservices and prevent maintainers from logging in as the tool. Disabled tools are archived and deleted after 40 days. Disabled tools can be re-enabled at any time prior to being archived and deleted.

The list of tools that have been disabled pending deletion can be found at toolforge:disabled-tools.

Backup Toolforge data

There are no user-accessible backups in Toolforge.

You should use a source or version control tool to preserve your code and make regular backups of data. Learn more.

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support

Chat in real time in the IRC channel #wikimedia-cloud ^connect or the bridged Telegram group
Discuss via email after you have subscribed to the cloud@ mailing list

Stay aware of critical changes and plans

Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
Read the News wiki page

Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read posts about Cloud Services on the Wikimedia Technical Blog. (Older posts can be read from the Cloud Services Blog.)