Portal:Cloud VPS/Admin/Projects lifecycle

From Wikitech
Jump to navigation Jump to search

This page describes how Cloud VPS projects are supposed to be created and deleted, i.e, the lifecycle. In the OpenStack world, project is known as tenant.

Project Requests

New project requests are made by the users in Phabricator. These requests are reviewed during the cloud services weekly meeting and if approved the on-call will fulfill the request.

After creating a project, when closing the Phabricator task please make sure to instruct the user to join the cloud-announce mailing list.

Creating a new project

The operator/engineer in charge of creating a new project should follow these steps.

Prerequisites:

  • Project names should be all lower case, if not some errors will happen when requesting the puppet certificate
  • Users have valid Developer accounts
  • Phabricator request task exists in phab:project/board/2875/ and has been approved
  • You have added the project to the list in Portal:Cloud VPS/Admin/Projects with the relevant contacts, tags, and link to the creation task.

Using horizon

Note that horizon task involving user management is very slow so please be patient.

  1. get the admin role in OpenStack
  2. jump to horizon https://horizon.wikimedia.org/
  3. switch to the admin project (top left drop-down menu)
  4. go to the identity zone https://horizon.wikimedia.org/identity/
  5. press the 'New project' button and follow the steps.
  6. amend initial quotas, if necessary (see section below)
  7. Communicate with the requestor in the pabricator task

Using CLI

  1. ssh to the corresponding control box (e.g. cloudcontrol1003.wikimedia.org, see deployments).
  2. elevate to root and source the novaenv.sh file (you can skip this step if you use wmcs-openstack instead of openstack)
    user@cloudcontrol1003:~ $ sudo su
    root@cloudcontrol1003:/home/aborrero# cd
    root@cloudcontrol1003:~# source novaenv.sh
    
  3. make sure the users requesting the project exists in LDAP
    root@cloudcontrol1003:~# openstack user list | grep -i user
    | user                                   | User name                                                                                |
    
  4. create the project (in this example, we are creating a project called newp)
    root@cloudcontrol1003:~# openstack --os-region-name eqiad1-r project create --enable --description "a dockerised wikibase infrastructure" newp
    +-------------+--------------------------------------+
    | Field       | Value                                |
    +-------------+--------------------------------------+
    | description | a dockerised wikibase infrastructure |
    | domain_id   | default                              |
    | enabled     | True                                 |
    | id          | newp                                 |
    | is_domain   | False                                |
    | name        | newp                                 |
    | parent_id   | default                              |
    +-------------+--------------------------------------+
    
  5. Modify initial quotas if required (see section below)
  6. Add initial users to roles:
    root@cloudcontrol1003:~# openstack role add --project newp --user aborrero  projectadmin
    root@cloudcontrol1003:~# openstack role add --project newp --user aborrero  user
    root@cloudcontrol1003:~# openstack role add --project newp --user tarrow  user
    root@cloudcontrol1003:~# openstack role add --project newp --user tarrow  projectadmin
    root@cloudcontrol1003:~# openstack role assignment list --names --project newp
    +--------------+---------------------------------+-------+--------------+--------+-----------+
    | Role         | User                            | Group | Project      | Domain | Inherited |
    +--------------+---------------------------------+-------+--------------+--------+-----------+
    | projectadmin | Arturo Borrero Gonzalez@Default |       | newp@Default |        | False     |
    | user         | Arturo Borrero Gonzalez@Default |       | newp@Default |        | False     |
    | projectadmin | novaadmin@Default               |       | newp@Default |        | False     |
    | user         | novaadmin@Default               |       | newp@Default |        | False     |
    | observer     | Novaobserver@Default            |       | newp@Default |        | False     |
    | projectadmin | Tarrow@Default                  |       | newp@Default |        | False     |
    | user         | Tarrow@Default                  |       | newp@Default |        | False     |
    +--------------+---------------------------------+-------+--------------+--------+-----------+
    
  7. Communicate with the requester in the phabricator task

Troubleshooting

If you get the following error below, it means that you have forgotten to "source novaenv.sh"

openstack: 'role' is not an openstack command. See 'openstack --help'.
Did you mean one of these?
  role add
  role create

Deleting a project

Project deletion tends to leave orphaned resources lying about. Eventually this should all be handled by Designate or Keystone hooks, but until then:

  1. Make sure this has been approved (a Phabricator task exists, etc)
  2. Log deletion in the proper SAL (e.g. in #wikimedia-cloud, !log <project> Removing per #T1234)
  3. Make sure there are no instances in the project. This can be done in Horizon or via the commandline:
    $ sudo wmcs-openstack server list --project <project>
    $ sudo wmcs-openstack server delete <instance id>
    
  4. Note if a custom flavor is present (these should be standardized after September 2020), and remove if no longer used outside of project:
    $ sudo wmcs-openstack flavor list
    $ sudo wmcs-openstack flavor delete <flavor name>
    
  5. Make sure the project is not configured for NFS modules/labstore/templates/nfs-mounts.yaml.erb.
  6. Make sure there are no dns zones allocated to the project. This can be done in Horizon or via the commandline:
    $ sudo wmcs-openstack zone list --sudo-project-id <project>
    $ sudo wmcs-openstack zone delete --sudo-project-id <project> <zone id>
    
  7. Delete any proxies the project may have via Horizon. Be sure to select the right project using the dropdown
  8. Delete any prefix- or project-wide puppet configurations via Horizon Here and Here. Be sure to select the right project using the dropdown
  9. Utilizing horizon, confirm no project specific config, rules, or resources remain.
  10. Finally, delete the project using Horizon or the commandline. If using Horizon, set your current project to 'Admin' and then delete the project using the Identity->Projects panel:
    $ sudo wmcs-openstack project delete <project>
    
  11. Update/resolve related phabricator tasks
  12. Remove the project from the list at Portal:Cloud VPS/Admin/Projects

If you need to drop some user involvement with a project:

user@cloudcontrol1003:~ $ sudo wmcs-openstack role remove --user someuser --project someproject projectadmin
user@cloudcontrol1003:~ $ sudo wmcs-openstack role remove --user someuser --project someproject user

Modifying project quotas

This can be easily done using the CLI:

  1. Make sure this has been requested by the project admins and approved by the WMCS team (a Phabricator task exists, etc)
  2. Modify quotas using the CLI, which is simply and straight-forward
    user@cloudcontrol1003:~$ sudo wmcs-openstack quota set --cores 120 --ram 320000 integration
    user@cloudcontrol1003:~$ sudo wmcs-openstack quota show integration
    +----------------------+-------------+
    | Field                | Value       |
    +----------------------+-------------+
    | cores                | 120         |
    | fixed-ips            | 200         |
    | floating-ips         | 50          |
    | floating_ips         | 0           |
    | injected-file-size   | 10240       |
    | injected-files       | 5           |
    | injected-path-size   | 255         |
    | instances            | 37          |
    | key-pairs            | 100         |
    | network              | 10          |
    | port                 | 50          |
    | project              | integration |
    | properties           | 128         |
    | ram                  | 320000      |
    | rbac_policy          | 10          |
    | router               | 10          |
    | secgroup-rules       | 100         |
    | secgroups            | 10          |
    | server_group_members | 10          |
    | server_groups        | 10          |
    | subnet               | 10          |
    | subnetpool           | -1          |
    +----------------------+-------------+
    
  3. Log the quota change in the project SAL
  4. Resolve the Phabricator task, letting requestors know

Floating ip quota

user@cloudcontrol1003:~$ sudo wmcs-openstack quota set --floating-ips 1 devtools
user@cloudcontrol1003:~$ sudo wmcs-openstack quota show devtools
+----------------------+----------+
| Field                | Value    |
+----------------------+----------+
| cores                | 16       |
| fixed-ips            | -1       |
| floating-ips         | 1        |
| health_monitors      | None     |
| injected-file-size   | 10240    |
| injected-files       | 5        |
| injected-path-size   | 255      |
| instances            | 8        |
| key-pairs            | 100      |
| l7_policies          | None     |
| listeners            | None     |
| load_balancers       | None     |
| location             | None     |
| name                 | None     |
| networks             | 100      |
| pools                | None     |
| ports                | 500      |
| project              | devtools |
| project_name         | devtools |
| properties           | 128      |
| ram                  | 32768    |
| rbac_policies        | 10       |
| routers              | 10       |
| secgroup-rules       | 100      |
| secgroups            | 40       |
| server-group-members | 10       |
| server-groups        | 10       |
| subnet_pools         | -1       |
| subnets              | 100      |
+----------------------+----------+

History

In previous versions of Cloud VPS all the projects lifecyle was handled via Wikitech itself, with some custom MediaWiki plugins to hook the Openstack API. This was eventually replaced by Horizon, which can handle all the operations by itself.

See also