Help:Toolforge/Envvars Service

From Wikitech
Jump to navigation Jump to search
The Toolforge Envvars Service is currently in Beta state, this means that it might change in non-backward compatible ways and/or suffer from outages and bugs. Although we encourage you to try it out and give feedback, please do not use it for critical tools until it becomes more stable and this banner is removed.

The Envvar Service is a very simple service that allows you to manage environment variables that will be available for your application when running (both webservices and jobs running in kubernetes are supported, built with the newer build service or using one of the provided images).

It allows an easy way to pass secrets to your application and tweak it's behavior per-environment.

Roadmap

Features that are already available

  • Create/update, delete and list environment variables.
  • Automatically inject environment variables to your application on startup (if you add or modify them, you'll have to restart your application).

Planned features that are not available yet

  • The database credentials used by ToolsDB and the wiki replicas will be automatically stored as envvars in the future.

Quickstart

Prerequisites

If you don't have a tool account yet, you need to create or join one. Detailed instructions are available at Help:Toolforge/Quickstart.

You should have a webservice or job running or ready to run. See Help:Toolforge/Build Service on how to build and start a webservice or job. Non-buildpack based webservices and jobs are also supported, for those see Help:Toolforge/Quickstart#Host your first tool on a tool account.

Creating a new environment variable

Once you have setup your toolforge account, you can ssh to the bastion and become your tool (using wm-lol tool as an example):

$ ssh myuser@login.toolforge.org
myuser@tools-sgebastion-10:~$ become wm-lol
tools.wm-lol@tools-sgebastion-10:~$

Now you can run toolforge envvars --help to see the available commands and syntax for each:

tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars --help
Usage: toolforge-envvars [OPTIONS] COMMAND [ARGS]...

  Toolforge command line

Options:
  -v, --verbose  Show extra verbose output. NOTE: Do no rely on the format of
                 the verbose output
  -d, --debug    show logs to debug the toolforge-envvars-* packages. For
                 extra verbose output for say build or job, see --verbose
  --version      Show the version and exit.
  --help         Show this message and exit.

Commands:
  create  Create/update an envvar.
  delete  Delete an envvar.
  list    List all your envvars.

For example, we can create a new environment variable (**note that the name for the envvar has to be all caps, just like a bash environment variable**):

tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars create API_KEY "my_api_key1"
name     value
API_KEY  my_api_key1

Now we have to start or restart the running webservice so it picks up the new environment variable (this example restarts a running buildservice based webservice):

tools.wm-lol@tools-sgebastion-10:~$ toolforge webservice buildservice restart

That's all! That environment variable is now available for the web service. :)

Updating an environment variable

Imagine that our api key has changed, and now we want to update it.

The process is exactly the same, the create command will update the variable if it already exists:

tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars list
name     value
API_KEY  my_api_key1


tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars create API_KEY "my_api_key2"
name     value
API_KEY  my_api_key2

Remember to restart your running webservices/jobs if you want the new value to be picked up!

tools.wm-lol@tools-sgebastion-10:~$ toolforge webservice buildservice restart


Common problems and solutions

Please add to this section any issues you encountered and how you solved them.

How to add the contents of a file to an enviroment variable?

Though it's not recommended, sometimes there's some files that you will want to put in an environment variable (ex. secret certificate keys). To do so you can use the same command, though make sure to separate the file contents with -- so the cli does not mistake them for cli options:

tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars create MY_CERT_KEY -- "$(cat secret.key)"

If you don't pass the --, you might get an error similar to:

Error: no such option: -----BEGIN RSA PRIVATE KEY-----`

Troubleshooting

We are actively building the debugging capabilities of the system, so they will be improved soon.

If you are unable to figure it out or found a bug, feel free to create a task (see the feedback section) or reach out to us on IRC/etc. (see communications section).

Known current limitations

Out of quota

There's a limited amount of envvars you can create (as of writing this the limit is 10, feel free to add your suggestion to this Phabricator task).

Feedback

If you try to use the Envvars Service, we would love to hear your feedback!

Currently the feedback is being tracked as part of the Help:Toolforge/Build_Service effort.

You can use this Phabricator template to report a bug, or this one for feature requests.

If you find this documentation page lacking in some way, or if you have more general comments, you can leave a comment in the Talk page.

For any other questions you can always reach us through the other communication channels listed at the bottom of this page.

Contributing

To contribute and/or follow up with the development of the project, take a look at the Ongoing Efforts page and the Contributing page.

History

Below are some historical discussions that led to its current design and implementation.

Communication and support

Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support
Stay aware of critical changes and plans
Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)