This page is currently a draft.
More information and discussion about changes to this draft on the talk page.
This page contains a system overview of Toolforge from the engineering point of view. Find here high level information of the different bits and pieces that make Toolforge what it is.
Toolforge is a Platform as a Service. It is meant to offer pre-made and managed computing facilities to its users.
At the core of its service, there is a computing engine. Well, as of this writing there are two, Grid Engine (to be deprecated) and Kubernetes. Users log in into Toolforge via SSH and deploy tools (mainly webservices and jobs) into one of the computing engines.
The About Toolforge user-oriented page contains more information on the basic aspects of the service.
All pieces of Toolforge are deployed inside a Cloud VPS project (or tenant) called
tools. The staging/development project is called
This section captures the most common use cases that Toolforge supports.
They key components
Information on the several key pieces that make Toolforge what it is.
This is where user accounts are stored.
Bastion servers are used by toolforge user as the entry point for the service. They are Cloud VPS virtual machines that allow SSH connections from the internet. Usual FQDNs are
A strong shared-system policy is enforced to ensure that users don't easily hog system resources (like CPU or RAM).
As of today, NFS is at the core of Toolforge. We use it for many purposes, including:
- to store toolforge user's home directories (remember, we allow SSH to the bastions)
- to store toolforge user's source code and distribute it to the computing backends (grid engine or kubernetes)
- to store toolforge user's credentials, like kubernetes TLS certs or wiki replicas database account credentials
- to store toolforge user's logs, generated at runtime from webservices and jobs
- to distribute wiki dumps
- to store toolforge tool temp files and similar
Please note that NFS is an old technology, and we have been working for years in reducing dependency on it with the ultimate goal of stop using NFS at all in the future.
We have a particular Kubernetes deployment, which is described in its own page.
There are a number of custom components deployed into Toolforge kubernetes, see Portal:Toolforge/Admin/Kubernetes/Custom_components.
RBAC and security
See main page: Portal:Toolforge/Admin/Kubernetes/RBAC_and_PSP.
See main page: Portal:Toolforge/Admin/Kubernetes/Jobs_framework.
Toolforge has a particular network setup, see Portal:Toolforge/Admin/Kubernetes/Networking_and_ingress.
See also Portal:Toolforge/Admin/Dynamicproxy.
There are a number of components that are part of the Toolforge offering beyond the key building blocks.
The entry point for account creation.
TODO. Talk here about.
- wiki replicas
TODO. We're trying to get rid of it.
- grid-only system