Jump to content

Help:Toolforge/Quickstart

From Wikitech

This page outlines the steps to get access to Toolforge and start working with files in a tool account.

Get access

If you already have one of these accounts, see the following sections of this page for how to proceed with Toolforge setup.

To access Toolforge, Cloud VPS, or other Wikimedia developer tools, you need the following two types of accounts:

Account Type Description Where to sign up
Wikimedia account Also known as "Wikimedia single user login (SUL)" account. Allows you to log into wikis like Wikipedia, MediaWiki.org, MetaWiki, and Wikitech (this wiki). Create Wikimedia account
Wikimedia developer account
  • Username (also called LDAP username) is used to log in on Gerrit, the Toolforge admin console, and other Wikimedia technical tools. In GitLab it is your displayed name, so you may want it to be your full name. You will not be able to change this later!
  • Shell username is used to log into Toolforge, other Cloud VPS or Wikimedia production hosts using SSH.
Create Wikimedia developer account

You need both of these types of accounts to use Toolforge.

If you already have a Wikimedia account but no developer account

You can create a developer account through the Toolforge admin console (Toolsadmin):

  1. Go to the registration page on Toolsadmin.
  2. In the first box press the Login using Wikimedia account button.
  3. You will be taken to a "Wikimedia account login" screen. Enter your Wikimedia account username and password.
  4. Once you are logged into your Wikimedia account, you will return to Toolsadmin. You will be prompted to create an LDAP username, a UNIX shell username, and password. You will also be prompted to enter your email address and agree to the terms of service and code of conduct for technical spaces.
    • The LDAP username will be used to log in to Wikitech, Gerrit and the Toolforge admin console.
    • The UNIX shell username will be used to interact with servers via ssh and for git with Gerrit.
  5. Submit the form to finish creating your Wikimedia developer account.

Because you used Toolsadmin to create your developer account, you don't need to complete any additional steps to link your accounts. Continue to request Toolforge membership.

If you already have both a Wikimedia account and a developer account

  1. Go to the the Toolforge admin console (Toolsadmin).
  2. In the first box press the Login using Wikimedia account button.
  3. You will be taken to a "Wikimedia account login" screen. Enter your Wikimedia account username and password.
  4. Go to the settings page in Toolsadmin to link your Wikimedia and Phabricator accounts to Toolforge.

Read and agree to terms of use

You must read and agree to the following terms to use Toolforge and other Wikimedia Cloud Services products:

Request Toolforge membership

  1. Submit a Toolforge membership request and wait for its approval.
  2. Your request will be reviewed, and you will receive confirmation within a week. You will be notified through your Wikitech user account.
  3. After you are added as a Toolforge member, log out and then log in again at toolsadmin.wikimedia.org. Doing this is required for your membership to take effect.

Connect to Toolforge servers using SSH

You can use SSH to log in to Toolforge servers after your developer account has been granted project membership.

Set up an SSH client and a key

  1. Install an SSH client, if you do not have one already. Most operating systems come with OpenSSH built in. Another popular client is PuTTY.
  2. Generate an SSH key pair. Each key pair consists of a public key and a corresponding private key. SSH uses public-key cryptography to securely authenticate you to servers.
  3. Add your public key to your developer account on this page of the Toolforge admin console. To do so, paste the content of your public key into the "New SSH Key" field and click "Add SSH key".
    • If you are using OpenSSH, you can find your public key in the file ~/.ssh/id_rsa.pub.
    • If you are using PuTTY, you can find your public key in the PuTTYgen window under "Public key for pasting into OpenSSH authorized_keys file".

Log in to Toolforge

  1. Use your SSH client to connect to Toolforge through the login.toolforge.org bastion host. Remember to use the shell username you picked when creating your Wikimedia developer account. You can see your username at toolsadmin.
    $ ssh -i <path-to-ssh-private-key> <shell-username>@login.toolforge.org
    
  2. On your first connection, your SSH client will ask you to verify the "fingerprint" of the server. Ensure that the fingerprint shown matches one of the current key fingerprints for login.toolforge.org.
  3. After you connect, the command prompt displays your username and the internal name of the host to which you are connected:
    <shell-username>@tools-bastion-13:~$
    

Create or join a tool account

After your Toolforge membership has been approved, create new tool accounts or ask other maintainers to add you to existing tool accounts. Each individual bot, web service or other tool should have a tool account.

Login and "become" your tool

After you create your tool account or join an existing tool account, and you have completed all the above setup steps:

  1. Log in to Toolforge
  2. From the command line, use the become command to switch to the tool account for the tool you want to work on:
    $ become <toolname>
    
    You should see the command prompt change to:
    tools.<toolname>@tools-sgebastion-10:~$
    

Refer to Help:Toolforge/Tool Accounts to learn more about what you can do when you have "become" your tool.

The bastion servers do not have compiler tools such as gcc installed. If you want to compile something, see Help:Toolforge/Troubleshooting#There_is_no_C/C++_compiler.

Create and manage files

Preferred way to interact with Toolforge is through a command-line interface. If you prefer to develop using IDEs like Visual Studio Code or Eclipse, you can mount the home directory of your tool onto your local Linux machine or virtual machine (VM) Learn more at Help:Toolforge/Tool Accounts.

You can access the files using SFTP clients such as Filezilla.

Toolforge is a shared environment, and by default all files you create will be public. This means that when you create files containing passwords or other secret information, you need to set correct file permissions to ensure it stays private.

Build and host your first tool

Now that you can connect to Toolforge and your tool account, you can proceed to build your tool in your preferred programming language.

The Toolforge Build Service is the currently recommended way to deploy new tools to Toolforge. Follow a tutorial to help you get started:

Tutorials for the older prebuilt image method are also available:

Some languages don't have yet a tutorial, but they are also supported. So in case this list gets out of sync, the updated list is the build service section for tutorials.

Need some inspiration? Use Toolhub to explore existing tools and get ideas.

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support
Stay aware of critical changes and plans
Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)