Help:Toolforge/Envvars Service
The Envvar Service is a very simple service that allows you to manage environment variables that will be available for your application when running (both webservices and jobs running in kubernetes are supported, built with the newer build service or using one of the provided images).
It allows an easy way to pass secrets to your application and tweak it's behavior per-environment.
Roadmap
Features that are already available
- Create/update, delete and list environment variables.
- Automatically inject environment variables to your application on startup (if you add or modify them, you'll have to restart your application).
Planned features that are not available yet
- The database credentials used by ToolsDB and the wiki replicas will be automatically stored as envvars in the future.
Quickstart
Prerequisites
If you don't have a tool account yet, you need to create or join one. Detailed instructions are available at Help:Toolforge/Quickstart.
You should have a webservice or job running or ready to run. See Help:Toolforge/Build Service on how to build and start a webservice or job. Non-buildpack based webservices and jobs are also supported, for those see Help:Toolforge/Quickstart#Host your first tool on a tool account.
Creating a new environment variable
Once you have setup your toolforge account, you can ssh to the bastion and become your tool (using wm-lol tool as an example):
$ ssh myuser@login.toolforge.org
myuser@tools-sgebastion-10:~$ become wm-lol
tools.wm-lol@tools-sgebastion-10:~$
Now you can run toolforge envvars --help to see the available commands and syntax for each:
tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars --help
Usage: toolforge-envvars [OPTIONS] COMMAND [ARGS]...
Toolforge command line
Options:
-v, --verbose Show extra verbose output. NOTE: Do no rely on the format of
the verbose output
-d, --debug show logs to debug the toolforge-envvars-* packages. For
extra verbose output for say build or job, see --verbose
--version Show the version and exit.
--help Show this message and exit.
Commands:
create Create/update an envvar.
delete Delete an envvar.
list List all your envvars.
For example, we can create a new environment variable (**note that the name for the envvar has to be all caps, just like a bash environment variable**):
tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars create API_KEY "my_api_key1"
name value
API_KEY my_api_key1
Now we have to start or restart the running webservice so it picks up the new environment variable (this example restarts a running buildservice based webservice):
tools.wm-lol@tools-sgebastion-10:~$ toolforge webservice buildservice restart
That's all! That environment variable is now available for the web service. :)
Updating an environment variable
Imagine that our api key has changed, and now we want to update it.
The process is exactly the same, the create command will update the variable if it already exists:
tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars list
name value
API_KEY my_api_key1
tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars create API_KEY "my_api_key2"
name value
API_KEY my_api_key2
Remember to restart your running webservices/jobs if you want the new value to be picked up!
tools.wm-lol@tools-sgebastion-10:~$ toolforge webservice buildservice restart
Common problems and solutions
Please add to this section any issues you encountered and how you solved them.
How to add the contents of a file to an enviroment variable?
Though it's not recommended, sometimes there's some files that you will want to put in an environment variable (ex. secret certificate keys). To do so you can use the same command, though make sure to separate the file contents with -- so the cli does not mistake them for cli options:
tools.wm-lol@tools-sgebastion-10:~$ toolforge envvars create MY_CERT_KEY -- "$(cat secret.key)"
If you don't pass the --, you might get an error similar to:
Error: no such option: -----BEGIN RSA PRIVATE KEY-----`
Troubleshooting
We are actively building the debugging capabilities of the system, so they will be improved soon.
If you are unable to figure it out or found a bug, feel free to create a task (see the feedback section) or reach out to us on IRC/etc. (see communications section).
Known current limitations
Out of quota
There's a limited amount of envvars you can create (as of writing this the limit is 10, feel free to add your suggestion to this Phabricator task).
Feedback
If you try to use the Envvars Service, we would love to hear your feedback!
Currently the feedback is being tracked as part of the Help:Toolforge/Build_Service effort.
You can use this Phabricator template to report a bug, or this one for feature requests.
If you find this documentation page lacking in some way, or if you have more general comments, you can leave a comment in the Talk page.
For any other questions you can always reach us through the other communication channels listed at the bottom of this page.
Contributing
To contribute and/or follow up with the development of the project, take a look at the Ongoing Efforts page and the Contributing page.
History
Below are some historical discussions that led to its current design and implementation.
Communication and support
Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:
- Chat in real time in the IRC channel #wikimedia-cloud connect, the bridged Telegram group, or the bridged Mattermost channel
- Discuss via email after you have subscribed to the cloud@ mailing list
- Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
- Read the News wiki page
Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself
Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)