Help:Toolforge/My first Rust tool

From Wikitech
Jump to navigation Jump to search

Overview

Rust is a relatively new language, but is rapidly gaining popularity and regularly voted as developers' favorite language. Rust prioritizes safety while enabling high performance (fearless concurrency).

This stub webservice is designed to get a sample Rust application installed onto Toolforge as quickly as possible. The application uses the Rocket framework.

The guide will teach you how to:

Getting started

Prerequisites

Skills

Accounts

Step-by-step guide

  • Step 1: Create a new tool account
  • Step 2: Enable Rust on your tool
  • Step 3: Create a basic Rocket webservice
  • Step 4: Compiling the application
  • Step 5: Launching the web service
  • Step 6: Troubleshooting

Step 1: Create a new tool account

  1. Create a new tool account.
    • For the examples in this tutorial, <TOOLNAME> is used to indicate places where your unique tool name is used in another command.
  2. SSH to login.toolforge.org.
    • If your local username is different from your shell account name on Toolforge, you will need to include your Toolforge shell account name in your ssh command (i.e. <shell_username>@login.toolforge.org).
  3. Run become <TOOLNAME> to change to the tool user.

Step 2: Enable Rust on your tool

The standard way to install Rust for development purposes is to use rustup. On Toolforge, rustup is already available, but needs to be enabled for your tool.

$HOME/.profile
. "/data/project/rustup/rustup/.cargo/env"
export RUSTUP_HOME=/data/project/rustup/rustup/.rustup

For this to take effect, you need to manually source your new profile. For future logins, this will automatically be taken care of.

$ source ~/.profile

You should now be able to see that Rust is installed and what the current version is:

$ rustc --version
rustc 1.56.1 (59eed8a2a 2021-11-01)

Step 3: Create a basic Rocket webservice

In its own words, Rocket is a web framework for Rust that makes it simple to write fast, secure web applications without sacrificing flexibility, usability, or type safety. If you are familiar with Python, you will notice some resemblances to the Flask web framework.

Rocket has its own guide and documentation, this tutorial will just cover the specifics of making it work on Toolforge.

First, you'll set up a project in the $HOME/www/rust directory for your application.

$ mkdir -p $HOME/www/rust
$ cd $HOME/www/rust
$ cargo init --name <TOOLNAME>
     Created binary (application) package

Then add the dependencies to be used to your Cargo.toml.

$HOME/www/rust/Cargo.toml
[package]
name = "<TOOLNAME>"
version = "0.1.0"
edition = "2021"

[dependencies]
# Helps with error handling
anyhow = "1.0"
# To connect to the wiki replicas
mysql_async = "0.28"
# Rocket framework
rocket = {version = "0.5.0-rc.1", features = []}
rocket_dyn_templates = {version = "0.1.0-rc.1", features = ["tera"]}
# Bridge for templates
serde = {version = "1.0", features = ["derive"]}
# Helps with connecting to wiki replicas
toolforge = {version = "5.0", features = ["mysql"]}

You can learn more about each crate by looking them up on docs.rs, which publishes documentation for nearly every published Rust crate.

Create a 'hello world' Rocket application. Each part of this file should be appropriately documented, explaining what it's doing.

$HOME/www/rust/src/main.rs
/*
This file is part of the Toolforge Rust tutorial

Copyright 2021 Kunal Mehta and contributors

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
#[macro_use]
extern crate rocket;

use anyhow::Result;
use mysql_async::prelude::*;
use rocket::http::Status;
use rocket_dyn_templates::Template;
use serde::Serialize;

/// The context needed to render the "index.html" template
#[derive(Serialize)]
struct IndexTemplate {
    /// The title variable
    title: String,
}

/// The context needed to render the "error.html" template
#[derive(Serialize)]
struct ErrorTemplate {
    /// The error message
    error: String,
}

/// Handle all GET requests for the "/" route. We return either a `Template`
/// instance, or a `Template` instance with a specific HTTP status code.
#[get("/")]
async fn index() -> Result<Template, (Status, Template)> {
    match get_latest_edit().await {
        // Got the title, render the "index" template
        Ok(title) => Ok(Template::render("index", IndexTemplate { title })),
        // Some error occurred when trying to query MySQL, render the "error"
        // template with a HTTP 500 status code
        Err(err) => Err((
            Status::InternalServerError,
            Template::render(
                "error",
                ErrorTemplate {
                    error: err.to_string(),
                },
            ),
        )),
    }
}

/// Get the latest edit via the enwiki database replica
async fn get_latest_edit() -> Result<String> {
    // Read from ~/replica.my.cnf and build a connection URL
    let db_url = toolforge::connection_info!("enwiki_p", WEB)?.to_string();
    // Create a new database connection pool. We intentionally do not use
    // Rocket's connection pooling facilities as Toolforge policy requires that
    // we do not hold connections open while not in use. So we open the
    // connection when we need to make a query and then immediately close it.
    let pool = mysql_async::Pool::new(db_url.as_str());
    // Open a connection and make a query
    let mut conn = pool.get_conn().await?;
    let resp: Option<String> = conn
        .query_first(
            r#"
SELECT
  rc_title
FROM
  recentchanges
WHERE
  rc_namespace = 0
ORDER BY
  rc_timestamp DESC
LIMIT
  1"#,
        )
        .await?;
    // Close the connections and pool (optional, this would happen
    // automatically when the variables are dropped)
    drop(conn);
    pool.disconnect().await?;
    // This unwrap is safe because we can assume that 1 row will always be returned
    Ok(resp.unwrap())
}

#[launch]
fn rocket() -> _ {
    rocket::build()
        .mount("/", routes![index])
        .attach(Template::fairing())
}

Note: The 'Hello World!' file above starts with a license header that places it under the GPLv3+ license.

Code on Toolforge must always be licensed under an Open Source Initiative (OSI) approved license. See the Right to fork policy for more information on this Toolforge policy.

Finally, you need to create two dynamic HTML templates for output.

$ mkdir -p $HOME/www/rust/templates
$HOME/www/rust/templates/index.html.tera
<!DOCTYPE HTML>
<html>
    <head>
        <title>My first Rust tool</title>
    </head>
    <body>
        <p>
            This is my first Rust tool.
        </p>
        <p>
            Did you know that the most recent edit to an article on the English Wikipedia
            was to <a href="https://en.wikipedia.org/wiki/{{title}}">{{title}}</a>?
        </p>
    </body>
</html>
$HOME/www/rust/templates/error.html.tera
<!DOCTYPE HTML>
<html>
    <head>
        <title>Error</title>
    </head>
    <body>
        <p>
            An error ocurred: {{error}}
        </p>
    </body>
</html>

These templates use the Tera template engine (again, if you're familiar with Python, these are similar to Jinja2/Django templates).

At this point, you should have a complete and working application.

Step 4: Compiling the application

The Toolforge login bastions (the server you SSHed into) are intended for basic interactive tasks, not running intensive programs, which unfortunately includes compiling Rust code. Instead, we'll compile our code using the "grid".

$ cd $HOME/www/rust
$ time jsub -N build -mem 2G -sync y -cwd cargo build --release
Your job 2997363 ("build") has been submitted
Job 2997363 exited with exit code 0.

real	5m51.013s
user	0m0.127s
sys	0m0.071s

If the build had failed, the error log containing the reason would be at $HOME/build.err. At this point, your application is fully compiled and ready to run.

Step 5: Launching the web service

By default, Rocket applications only listen on 127.0.0.1 (localhost), meaning that they only accept connections from the same server. From a security perspective, this is a good default, but you intentionally want to expose our application to everyone, specifically the Toolforge proxy that routes requests to your tool.

Create a small wrapper shell script to start your Rocket application with the correct settings.

$HOME/www/rust/run-webserver.sh
#!/bin/sh
cd ~/www/rust
ROCKET_ADDRESS=0.0.0.0 ROCKET_LOG_LEVEL=normal ./target/release/<TOOLNAME>

Note: The name of the file specifically comes from whatever you have in Cargo.toml under the package.name field. If you're unsure, look inside the $HOME/www/rust/target/release/ folder to see what it's called.

Make sure the file is executable.

$ chmod +x $HOME/www/rust/run-webserver.sh

Next, create a template file telling the webservice system how to start your application (note: this goes directly in your $HOME directory).

$HOME/service.template
backend: kubernetes
type: golang111
extra_args:
  - www/rust/run-webserver.sh

You may have noticed that this confusingly uses the "golang111" image. Your Rocket application doesn't actually require any dependencies, but you need to pick some webservice type, so golang111 was picked since there aren't that many actual golang web services. In the future a "Rust" type might be added.

Actually start the webservice

$ webservice start
Starting webservice...

Visit https://<TOOLNAME>.toolforge.org/ to see your new Rust application in action! Congratulations!

Step 6: Troubleshooting

The logs and output emitted by the Rocket application are available through Kubernetes.

$ kubectl get pods
NAME                                  READY   STATUS    RESTARTS   AGE
<TOOLNAME>-7f7b847466-fvkq2   1/1     Running   0          3m37s
$ kubectl logs <TOOLNAME>-7f7b847466-fvkq2
...
Rocket has launched from http://0.0.0.0:8000
GET / text/html:
   >> Matched: (index) GET /
   >> Outcome: Success
   >> Response succeeded.
...

If the application panics or runs into other errors, you should find the logs here.

Resources

Next steps

Now that your Rust tool using Rocket is set up, here are some next steps to consider:

Communication and support

We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.

Communicate with us
Way Connect Best for
Phabricator Workboard #Cloud-Services Task tracking and bug reporting
IRC Channel #wikimedia-cloud connect
Telegram bridge
mattermost bridge
General discussion and support
Mailing List cloud@ Information about ongoing initiatives, general discussion and support
Announcement emails cloud-announce@ Information about critical changes (all messages mirrored to cloud@)
News wiki page News Information about major near-term plans
Cloud Services Blog Clouds & Unicorns Learning more details about some of our work
Wikimedia Technical Blog techblog.wikimedia.org News and stories from the Wikimedia technical movement