User:Legoktm/Rust on Toolforge
|Interested in working toward advancing Rust in the Wikimedia movement? Join the Wikimedia Rust developers user group today!|
Rust is not currently officially supported on Toolforge. Here are some tips:
Compiling Rust code
When it comes to building your Rust project, you have two primary options. The first and depending upon your hardware, faster, option is to build it on your own machine. It is recommended to do this in a Debian 10/Buster environment (in a VM, container, etc.), to make sure you're linking against compatible system libraries (usually
libssl). A brief test of building on Fedora 31 also worked. You may want to look into cross-compiling if you're on macOS/Windows. Ideally we would be able to use a Toolforge docker image for the building process (even on our own machines) but that's not practical yet since all of the potential images are missing at least
If you want to build your project on Toolforge, you'll need to use the grid engine. First, install rustup in your tool. Since you only want to build code, you only need the minimal profile:
curl https://sh.rustup.rs -sSf | sh -s -- --profile minimal. If you want nightly, pass
Assuming your project is cloned to
~/www/rust, try something like:
jsub -N build -mem 2G -sync y cargo install --path ~/www/rust --locked. Depending on what exactly you're building, you may need to bump the memory limit. The
-sync y option allows for easy scripting once the build completes, e.g. running
Your code will need to start a webserver on port 8000 (you can hardcode this) and listen on 0.0.0.0.
Once your binary is on Toolforge, start a webservice via
webservice golang111 start .cargo/bin/$NAME. Add extra options like
--canonical as necessary. I recommend putting these options in a
~/service.template so you don't need to remember them each time:
backend: kubernetes canonical: True # Only if your tool previously used the tools.wmflabs.org/$TOOL domain scheme type: golang111 extra_args: - .cargo/bin/$NAME
Note that while any "generic" webservice can be used, Legoktm recommends using the
golang111 so we can more easily track Rust projects.
Tools using Rust
If a tool you wrote or use is using Rust on Toolforge, please add to this list!
- Magnus Manske
- QuickStatements (the "run in background" bot doing the major gruntwork for now, but eventually all of the server-side code)
- SourceMD (currently deactivated, needs some work, time- rather then tech-constrained)
- templatehoard: the
dump-parsed-templatessubcommand in enwikt-dump-rs generates template dumps in CBOR format. It uses a JSON file of redirects in the template namespace generated by an example program from the
redirect.sqldump files using the parse-mediawiki-sql library. templatehoard-server serves the non-static site, which provides a search page for all the template dumps and another for the IPA template.
- digero: process-with-lua generates indices of the language headers found in each entry.
- templatehoard: the
- socksfinder (builds an inverted index from a MediaWiki dump and exposes it through a webservice for the user to find sockpuppets ; only the service part is run on Toolforge for now, though I'd like to schedule the index build part on SGE as well ; compile is done in an LXC container on my own box)