Help:Toolforge/Migrate build service job to push to deploy
This tutorial explains how to initialize your Toolforge configuration from an existing Toolforge job that uses an image built with the Build Service, and then configure Push-to-Deploy so that future Git pushes automatically trigger deployments.
If you already use the Build Service, most of the information needed to create the tool configuration can be generated automatically from your existing job.
The job we'll be migrating is a simple Toolforge job named recent-changes, that fetches the most recent changes on English Wikipedia.
The migration has two main stages:
- Create your tool configuration: Configure your tool to build and deploy your existing Build Service job and verify that the component works.
- Configure and enable Push-to-Deploy: Configure your Git repository to trigger deployments automatically when you push changes.
Before you begin
This tutorial assumes that:
- you have access to the Toolforge tool account
- you already have a working Toolforge job
- the job uses an image built with the Toolforge Build Service
- you can commit and push changes to the job source repository
Create your tool configuration
Become your tool
Once you logged into login.toolforge.org, become your tool before running Toolforge commands in this tutorial:
myuser@tools-bastion-15:~$ become <TOOL NAME>
Replace <TOOL NAME> with the name of your Toolforge tool.
Inspect your existing job
First, inspect the current job configuration:
tools.komla-test3@tools-bastion-15:~$ toolforge jobs show recent-changes
For example, the recent-changes job looks similar to this:
Job name: recent-changes
Command: ./recent-changes-job.sh
Job type: scheduled: @daily
Image: tool-komla-test3/recent-changes:latest
File log: no
Emails: none
Resources: default
Mounts: all
Retry: no
Timeout: yes: 3600s
Status: Pending for 50m59s
We see that we have one scheduled job named recent-changes that runs daily and is using the tool-komla-test3/recent-changes:latest image. When the image is in the format tool-<yourtool>/<imagename>:latest it means that it's a Build Service built image. Make sure that the job you choose uses one of these images.
Once you know which image it uses, you can also inspect previous builds:
tools.komla-test3@tools-bastion-15:~$ toolforge build list
build_id status start_time end_time source_url image_name ref envvars use_latest_versions use_deprecated_versions destination_image
komla-test3-buildpacks-pipelinerun-wbcvt ok 2026-09-30T14:20:08Z 2026-09-30T14:20:43Z https://gitlab.wikimedia.org/toolforge-repos/komla-test3 recent-changes main N/A False False tools-harbor.wmcloud.org/tool-komla-test3/recent-changes:latest@sha256:dcd2241e25a8fa6c45c918623b1115fb189227608e961f3064865eed228fd55e
...
You can check the last build by running:
tools.komla-test3@tools-bastion-15:~$ toolforge build show
Build ID: komla-test3-buildpacks-pipelinerun-wbcvt
Start Time: 2026-09-30T14:20:08Z
End Time: 2026-09-30T14:20:43Z
Status: ok
Message: Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0
Parameters:
Source URL: https://gitlab.wikimedia.org/toolforge-repos/komla-test3
Image Name: recent-changes
Ref: main
Envvars: N/A
Use latest versions: False
Use deprecated versions: False
Destination Image: tools-harbor.wmcloud.org/tool-komla-test3/recent-changes:latest@sha256:dcd2241e25a8fa6c45c918623b1115fb189227608e961f3064865eed228fd55e
Note that the Destination Image matches the image in our job.
Generate a Tool configuration
You can start from scratch, or try generating an example configuration from your existing jobs.
Run:
tools.komla-test3@tools-bastion-15:~$ toolforge components config generate
Warning: Note that this config is an autogenerated example, please double check and validate before using it
config_version: v1beta1
components:
recent-changes:
build:
repository: https://gitlab.wikimedia.org/toolforge-repos/komla-test3
ref: main
run:
command: python recent-changes.py
schedule: '@daily'
timeout: 3600
Note the warning on the first line, make sure to verify the generated configuration. It will also show all the jobs it was able to detect (continuous and scheduled jobs that use build-service images), you can add them one by one, no need to add them all at the same time.
For details on what to put in the configuration, see Help:Toolforge/Deploy your tool#Creating your tool configuration, but here's a high level overview of the config we got:
# Version of the configuration schema, not mandatory but helps to know what schema to use
config_version: v1beta1
# Here we list each of our components, each component is made of a `build` (like an image) and a `run` (like a job)
components:
# Our first component, called `recent-changes`
recent-changes:
build:
# Public url to the git repository
repository: https://gitlab.wikimedia.org/toolforge-repos/komla-test3
# Branch
ref: main
# This is the job to run
run:
# same as `--command` in `toolforge jobs`
command: python recent-changes.py
# this would be the `--schedule`
schedule: '@daily'
# and this would be the `--timeout`
timeout: 3600
Note that you can also run a continuous job (ex. if you don't pass schedule) or even a webservice/published continuous job, like the example in the docs.
The fields match the options that are passed on the toolforge jobs cli, with minor changes like --mem becomes memory:, you can see the full list in the API docs and a jsonschema version in the repo (that you can use for your editor autocomplete).
Save and review the generated configuration
You can save this configuration in a file, and edit live (we will see when configuring Push-to-Deploy how to tell it to fetch it from your git repository instead):
tools.komla-test3@tools-bastion-15:~$ toolforge components config generate > toolforge.yaml
tools.komla-test3@tools-bastion-15:~$ vim toolforge.yaml # or use your preferred editor
Make sure to check that the parameters set there match the ones for your job, and add/remove/modify them if they do not. Delete any components that you don't wont to configure now if there's more than one.
Create the configuration in Toolforge
Once reviewed, you can create the configuration for your tool with:
tools.komla-test3@tools-bastion-15:~$ toolforge components config create toolforge.yaml
# You can double check your current config with
tools.komla-test3@tools-bastion-15:~$ toolforge components config show
tools.komla-test3@tools-bastion-15:~$ toolforge components config show
Warning: You are using a beta feature of Toolforge.
config_version: v1beta1
components:
recent-changes:
build:
repository: https://gitlab.wikimedia.org/toolforge-repos/komla-test3
ref: main
run:
command: python recent-changes.py
schedule: '@daily'
timeout: 3600
Create a deployment to test it worked
Now that you have a configuration, you can create deployments. A deployment will create all the images needed by all your jobs, and update/create your jobs if needed.
tools.komla-test3@tools-bastion-15:~$ toolforge components deployment create
Deployment ID: 20261007-170802-v8e5r8zyz8
Created: 20261007-170802
Builds:
recent-changes(pending): no-id-yet
Runs:
recent-changes(pending):
For other options during deployment see Help:Toolforge/Deploy your tool#Creating a deployment.
Now you can check the status of the deployment, it will show you the builds that it triggered, and the runs it created (when it does). You can keep checking until it passes:
tools.komla-test3@tools-bastion-15:~$ toolforge components deployment show
Deployment ID: 20261007-170942-psayjz3xf9
Created: 20261007-170942
Status: successful
Long status:
Finished at 2026-10-07 17:10:20.063507+00:00
Builds:
recent-changes(successful): id:komla-test3-buildpacks-pipelinerun-jthrj You can see the logs with `toolforge build logs komla-test3-buildpacks-pipelinerun-jthrj`
Runs:
recent-changes(successful): created or updated job recent-changes, [info](Job recent-changes was updated in storage and runtime)
Tool config:
components:
recent-changes:
build:
ref: main
repository: https://gitlab.wikimedia.org/toolforge-repos/komla-test3
run:
command: python recent-changes.py
schedule: '@daily'
timeout: 3600
config_version: v1beta1
And done! Your job should now be running with the newly built image, you will notice that now the image has a sha256:... trailing the image name, that means that it worked:
tools.komla-test3@tools-bastion-15:~$ toolforge jobs show recent-changes
...
+---------------+----------------------------------------------------------------------------------------------------------------+
| Image: | tool-komla-test3/recent-changes:latest@sha256:b88c7d086041dfdc1ea91505ae65177aa6bae42adc0077049a97b1e32ac6c8d0 |
+---------------+----------------------------------------------------------------------------------------------------------------+
...
Configure and enable Push-to-Deploy
Push-to-Deploy allows a CI runner to trigger a deployment after changes are pushed to your Git repository.
Below are the steps for a basic configuration. Visit Triggering a deployment from your CI runner for more details.
Add the configuration to your source repository
This step is not strictly needed, but it enables not having to ssh to toolforge to change your configuration, and to keep the changes to it tracked in a Git repository.
The tool configuration can be hosted anywhere that is publicly accessible thorugh http/https.
For this tutorial, save the generated configuration as toolforge.yaml in the same Git repository as the source code. Keeping the source code and deployment configuration together makes it easier to track changes to both in Git.
You can use any publicly accessible Git repository. For Toolforge projects, we recommend using a repository in the Wikimedia GitLab toolforge-repos namespace (you can create them from the toolsadmin page of your tool).
For example:
my-project/
├── Procfile
├── requirements.txt
├── source-files...
└── toolforge.yaml
There's one change you need to make to your configuration before commiting it, we have to add source_url:
config_version: v1beta1
# We only added the nex entry
# This url should point to the "raw" version of your file in your hosting, this example uses gitlab
source_url: https://gitlab.wikimedia.org/toolforge-repos/komla-test3/-/raw/main/toolforge.yaml?ref_type=head
components:
recent-changes:
build:
repository: https://gitlab.wikimedia.org/toolforge-repos/komla-test3
ref: main
run:
command: python recent-changes.py
schedule: '@daily'
timeout: 3600
Commit and push the file:
mylaptop$ git add toolforge.yaml
mylaptop$ git commit -m "Add Toolforge configuration"
mylaptop$ git push
Update Tool configuration
Once toolforge.yaml is publicly accessible, you can update your tool configuration with a single command, using the url you added as source_url:
tools.komla-test3@tools-bastion-15:~$ curl 'https://gitlab.wikimedia.org/toolforge-repos/komla-test3/-/raw/main/toolforge.yaml?ref_type=heads' | toolforge components config create
You will see information saying configuration created or updated successfully.
This avoids having to clone the repository or download toolforge.yaml on the Toolforge bastion at all.
From now on Toolforge will update your configuration at the start of every deployment, pulling it from that URL, so from now on you don't need to change it manually from the bastion anymore.
Configure your CI to trigger a deployment
The following is a minimal example for Wikimedia GitLab. For additional options and details, see Triggering a deployment from your CI runner.
Create a deploy token
While logged in as your tool account, create a deploy token:
tools.komla-test3@tools-bastion-15:~$ toolforge components deploy-token create
Warning: You are using a beta feature of Toolforge.
Token: SOME-LONG-UUID-KIND-OF-STRING
Created: 2026-09-30T14:10:28.538502Z
The command displays a token. Treat this token as a secret. Do not commit it to your repository.
Add the deploy token to GitLab
In your GitLab repository, go to:
Settings → CI/CD → Variables
Create a variable named:
TOOLS_DEPLOY_TOKEN
Set its value to the deploy token created in the previous step.
For security, make the variable hidden, protected, and masked where appropriate.
Add a GitLab CI configuration
Create a .gitlab-ci.yml file in the repository containing:
include:
- project: "repos/cloud/cicd/gitlab-ci"
file: "toolforge-cd/deploy-to-toolforge.yaml"
The included Toolforge CI configuration handles triggering the deployment.
This minimal configuration assumes that the GitLab project name and Toolforge tool name are the same. If they are different, or if you need to customize the deployment process, see Triggering a deployment from your CI runner.
Commit and push .gitlab-ci.yml to the repository.
When a commit is pushed or merged to the default branch, GitLab CI will use the deploy token to request a new deployment for komla-test3.
You should see now when you pushed that change, that a new Gitlab pipeline triggered and did a deploy (see the screenshot), congratulations!
The setup gives the CI runner the credentials it needs to request a deployment for your tool.
Avoid independently managing the migrated job
After migrating your job, any manual changes that you make will be overwritten by Toolforge on the next deployment. So if you want them to last remember to change your Toolforge configuration in your git repository instead.
What changed?
Before migration, the deployment workflow is approximately:
Git push
↓
Ssh login.toolforge.org
↓
become <mytool>
↓
toolforge build start ...
↓
toolforge jobs delete/run/restart ...
After migrating to using a Tool configuration and cnfiguring Push-to-Deploy, the workflow becomes:
Git push
↓
CI triggers deployment
↓
Toolforge builds your images
↓
Toolforge updates your jobs
This allows you to change your code, and get it deployed without any more action on your side than git push.
Troubleshooting
Check the generated configuration
If the generated configuration does not look correct, do not deploy it immediately.
Compare it with:
myuser@tools-bastion-15:~$ toolforge jobs list -o long
myuser@tools-bastion-15:~$ toolforge build list
Correct toolforge.yaml before creating the Tool configuration.
Check the Tool configuration
Run:
myuser@tools-bastion-15:~$ toolforge components config show
Verify that the repository, ref, component name, command, schedule, and other settings are correct.
Check deployment history
Run:
myuser@tools-bastion-15:~$ toolforge components deployment list
Use the deployment information to determine whether the build or deployment failed.
Check your CI pipeline
If pushing a change does not create a deployment, check your repository's CI pipeline.
Verify that:
- the CI job ran after the Git push;
- the CI job completed successfully;
- the credentials required to trigger a Toolforge deployment are configured correctly.
See Triggering a deployment from your CI runner for the current setup instructions.
Rebuild the image manually
By default, a new deployment will not rebuild an image if the source code for it did not change. Sometimes for troubleshooting, you want force build the image even when the code did not change, for that you can run:
$ toolforge components deployment create --force-build
This creates a deployment that bypasses the git hash check, and rebuilds the image even if the code did not change.
Force recreating the job on deployment
Also by default, a deployment will not recreate a job if the image it uses did not change (did not get built) and the job already exists with the same configuration, but sometimes you want to force it to delete the job and createn anew, for that you can pass:
$ toolforge components deployment create --force-run
Failing to generate config from a running job
Make sure the job is using a build service image and not just a standard Toolforge image. Note that not all the jobs are supported (ex. one-off jobs or webservices started with toolforge webservice).
Communication and support
Support and administration of the WMCS resources is provided by Wikimedia Foundation staff and Wikimedia movement volunteers. Please reach out with questions and join the conversation:
- Chat in real time in the IRC channel #wikimedia-cloud connect or the bridged Telegram group
- Discuss via email after you have subscribed to the cloud@ mailing list
- Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
- Read the News wiki page
Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself
Read posts about Cloud Services on the Wikimedia Technical Blog. (Older posts can be read from the Cloud Services Blog.)