Help:Toolforge/Rust
Rust is a relatively new language, but is rapidly gaining popularity and regularly voted as developers' favorite language. Rust prioritizes safety while enabling high performance (fearless concurrency).
On Toolforge the use of Rust is supported by the toolforge admins when using the build service (see the tutorial), or by fellow community members who are also using Rust.
New to Rust? Check out the My first Rust tool tutorial!
Rust toolchain
The standard way to install Rust for development purposes is to use rustup. On Toolforge, rustup is already available, but needs to be enabled for your tool.
. "/data/project/rustup/rustup/.cargo/env"
export RUSTUP_HOME=/data/project/rustup/rustup/.rustup
For this to take effect, you need to manually source your new profile (source $HOME/.profile). For future logins, this will automatically be taken care of.
You should now be able to see that Rust is installed and what the current version is:
$ rustc --version
rustc 1.56.1 (59eed8a2a 2021-11-01)
If you need a different version of Rust (e.g. nightly), please ask one of the "rustup" tool maintainers to install it.
Compiling Rust code
To compile Rust code in Toolforge, use the jobs framework. This is recommended because it ensures you will link against compatible versions of system libraries (typically libc and libssl) and also keeps the Rust source on Toolforge, making it easier to fork. Please note that this only works with tool accounts.
$ toolforge jobs run build --command "bash -c 'source ~/.profile && cd ~/my/rust/code && cargo build --release'" --image python3.11 --mem 2G --cpu 3
If the build fails, the error log containing the reason will be at $HOME/build.err.
Local compilation
You can also compile code locally and copy the resulting binary to Toolforge. Depending on your hardware, this might be faster.
It is recommended to do this in a Debian environment (in a VM, container, etc.) to make sure you're linking against compatible system libraries (usually libc and libssl). Windows/macOS users may want to look into cross-compiling, targetting either x86_64-unknown-linux-gnu or x86_64-unknown-linux-musl.
Running a webservice
Your code will need to start a webserver on port 8000 (you can hardcode this) and listen on 0.0.0.0.
Once you have a binary on Toolforge (in this example we'll use .cargo/bin/<name>, it could also be www/rust/target/release/<name>), start a webservice via webservice ruby3.1 start .cargo/bin/<name>. Add extra options like --canonical as necessary. You can put these options in a $HOME/service.template so you don't need to remember them each time:
backend: kubernetes
type: ruby3.1
extra_args:
- .cargo/bin/<name>
Then a plain webservice start will start up your webservice! It is important to note that this command will run in the context of your home directory, so if it needs to run in a subdirectory to find templates or other resources, write a small shell script that changes the path:
#!/bin/bash
cd ~/www/rust
./target/release/<name>
Note that while any "generic"-type webservice can be used, Legoktm recommends using the ruby3.1 so we can more easily track Rust projects. In the future a Rust type might be added.
Running a job
Using the toolforge-jobs service, it is recommended you use the bookworm container. It only has basic system libraries like libc, libssl, and libmysqlclient available, and most Rust programs won't need more than that.
Tools using Rust
If a tool you wrote or use is using Rust on Toolforge, please add to this list!
- Magnus Manske
- PetScan
- QuickStatements (the "run in background" bot doing the major gruntwork for now, but eventually all of the server-side code)
- Listeria bot
- SourceMD (currently deactivated, needs some work, time- rather then tech-constrained)
- Legoktm
- toolforge:logo-test, toolforge:newusers, toolforge:shorturls, uatu, ircservserv, planning to port other stuff
- Erutuon
- templatehoard: the
dump-parsed-templatessubcommand in enwikt-dump-rs generates template dumps in CBOR format. It uses a JSON file of redirects in the template namespace generated by an example program from thepage.sqlandredirect.sqldump files using the parse-mediawiki-sql library. templatehoard-server serves the non-static site, which provides a search page for all the template dumps and another for the IPA template. - digero: process-with-lua generates indices of the language headers found in each entry.
- templatehoard: the
- Arkanosis
- socksfinder (builds an inverted index from a MediaWiki dump and exposes it through a webservice for the user to find sockpuppets; the service part is run using Toolforge webservice's Kubernetes backend and the index build part is scheduled twice a month using Toolforge jobs framework on Kubernetes; compile is done in an LXC container on my own box)
- User:Kosta Harlan
- Mmarx
- SQID now uses a rust helper to do the offline dump analysis
- Count Count
- Spamcheck: Both the link recorder and the webpage are written in Rust.
- Multiple tools for dewiki: Tool for checking RfA votes, bot for resolving range blocks on de:Wikipedia:Vandalismusmeldung, etc.
See also
Communication and support
Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:
- Chat in real time in the IRC channel #wikimedia-cloud connect or the bridged Telegram group
- Discuss via email after you have subscribed to the cloud@ mailing list
- Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
- Read the News wiki page
Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself
Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)