Portal:Cloud VPS/Admin/Openstack cli

This document is intended for use by WMCS admins, specifically those with shell accounts on cloudcontrol hosts. Some of these instructions may be accurate or useful elsewhere and some may not.

openstack

The canonical command line interface for OpenStack is openstack. It ought to be able to reach every endpoint supported in our cloud via a plugin-system that allows additional subcommands to be injected project-by-project.

To see available subcommands, run 'openstack help'.

Your workflow will either involve running openstack as root, or running 'sudo wmcs-openstack. All subsequent commands and flags should be identical.

wmcs-openstack

The wmcs-openstack command is a very thin wrapper around the openstack command. It sets up default auth as the 'novaadmin' user and then dispatches all remaining flags and subcommands to 'openstack'.

Your workflow will either involve running openstack as root, or running 'sudo wmcs-openstack'. All subsequent commands and flags should be identical.

project-specific tools

Prior to the creation of the integrated openstack interface, each project had its own commandline tool. Occasionally our documentation will contain obsolete references to these tools, suggesting that you run e.g. 'nova server-list' rather than 'openstack server list'. Syntax is subtly and unpredictably different between the old and new tools, and the 'openstack' tool should support everything that can be done with the project-specific tools. Please correct and update documentation whenever you find references to these older tools.

Auth

Gaining a useful OpenStack token is messy due to granularity that openstack supports. Typically you will have to set an endpoint, a username, a password, an auth project, and possibly an auth domain. To avoid finger cramps we have puppet install several expedited systems for auth shortcuts.

command-line flags

If you type it all out, an authorized commandline looks like this:

root@cloudcontrol1005:~# openstack server list --project admin-monitoring  --os-username novaadmin  --os-password <redacted> --os-auth-url https://openstack.eqiad1.wikimediacloud.org:25357/v3 --os-user-domain-id default --os-project-id admin
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| ID                                   | Name                      | Status | Networks                               | Image                | Flavor                |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| b7f8711c-769a-4753-81d4-33161a88be5a | fullstackd-20230605013424 | ACTIVE | lan-flat-cloudinstances2b=172.16.5.176 | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+

Values for all of the above can be found in puppet-generated yaml files found in /etc. For example, the novaadmin creds are in /etc/novaadmin.yaml. Here's a longer list:

novaadmin.yaml: For powerful, can-do-almost-anything creds
novaobserver.yaml: For read-only activies. These credentials are public and installed on most VMs.
wmflabsorg-domainadminenv.yaml: Credentials for a service user with permission to create and delete DNS zones. Most often used by acme-chief for certificate validation.
oss-canary.yaml: Stores the credentials used by the nova-fullstack monitoring test suite

The following credential sets are not currently useful and may be removed:

keystoneadmin.yaml: provides domain-scoped access. Probably not useful as long as scoped auth is disabled.
ossystemadmin.yaml: provides system-scoped access. Probably not useful as long as scoped auth is disabled.
ossystemobserver.yaml: provides system-scoped access for read-only endpoints. Probably not useful as long as scoped auth is disabled.

shell environment

For each of the yaml files listed in command-line flags there is a corresponding shell script that will inject each of the listed credentials into the environment. For example:

root@cloudcontrol1005:/# source /root/novaenv.sh 
root@cloudcontrol1005:/# openstack server list --project admin-monitoring
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| ID                                   | Name                      | Status | Networks                               | Image                | Flavor                |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| 133cd53f-a47d-4188-8864-c1a0bb18913d | fullstackd-20230605185923 | BUILD  |                                        | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
| b7f8711c-769a-4753-81d4-33161a88be5a | fullstackd-20230605013424 | ACTIVE | lan-flat-cloudinstances2b=172.16.5.176 | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+

For historical reasons, '/root/novaenv.sh' corresponds to novaadmin.yaml. The other scripts are in /usr/local/bin or /root depending on their level of access.

clouds.yaml

the openstack tool increasingly expects to read credentials from clouds.yaml. There are a few auth edge-cases that can only be implemented via a cloud config, and likely to be more in the future. For that reason, this is the preferred auth method for the commandline.

The new way to provide credentials to the openstack cli is via the clouds.yaml config file. There are two such files: read/write credentials are stored in /root/.config/openstack/ and read-only credentials in /etc/openstack/clouds.yaml.

There is a set of clouds.yaml credentials that corresponds to each of the yaml files mentioned above, which the same title. To invoke those credentials, use --os-cloud on the cli:

root@cloudcontrol1005:~# openstack server list --os-cloud novaadmin --project admin-monitoring
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| ID                                   | Name                      | Status | Networks                               | Image                | Flavor                |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| 7af8e828-3159-4b25-874f-1fe19a1fded9 | fullstackd-20230605191933 | ACTIVE | lan-flat-cloudinstances2b=172.16.6.111 | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
| b7f8711c-769a-4753-81d4-33161a88be5a | fullstackd-20230605013424 | ACTIVE | lan-flat-cloudinstances2b=172.16.5.176 | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+

or set a reference to the clouds section in your environment:

root@cloudcontrol1005:~# export OS_CLOUD=novaadmin
root@cloudcontrol1005:~# openstack server list --project admin-monitoring
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| ID                                   | Name                      | Status | Networks                               | Image                | Flavor                |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+
| 7af8e828-3159-4b25-874f-1fe19a1fded9 | fullstackd-20230605191933 | ACTIVE | lan-flat-cloudinstances2b=172.16.6.111 | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
| b7f8711c-769a-4753-81d4-33161a88be5a | fullstackd-20230605013424 | ACTIVE | lan-flat-cloudinstances2b=172.16.5.176 | debian-11.0-bullseye | g3.cores1.ram2.disk20 |
+--------------------------------------+---------------------------+--------+----------------------------------------+----------------------+-----------------------+

Project selection

Project selection is handled inconsistently among the various backends. Some workflows expect the project as a named argument, like

andrew@cloudcontrol1005:~$ sudo wmcs-openstack server list --project testlabs

Many of our workflows assume specifying a project in the environment, like

andrew@cloudcontrol1005:~$ OS_PROJECT_ID=testlabs sudo wmcs-openstack zone list

That last command does not work with clouds.yaml, as it tends to override any environment settings.

It should be possible to accomplish everything within openstack using a combination of clouds.yaml settings and named arguments, but it requires some hunting. For example, the equivalent to the above 'zone list' command is

andrew@cloudcontrol1005:~$ sudo wmcs-openstack zone list --os-cloud novaadmin --sudo-project-id testlabs

Despite the inconsistency in arg names, that last option is the preferred method. If you find a command that does not permit specifying project as a named argument, please open a bug.

Related tasks

https://phabricator.wikimedia.org/T330759 Modernize openstack RBAC

https://phabricator.wikimedia.org/T276018 Investigate new roles and policies in openstack Xena