Jump to content

Help:Cloud VPS Instances

From Wikitech
(Redirected from Help:Instances)

This page contains information on how to set up a Cloud VPS instance.

What is a Cloud VPS instance?

An instance is a virtual machine (VM). Every instance belongs to a Cloud VPS project.

Difference between a Cloud VPS project and an instance

  • A Cloud VPS project can be defined as the group of users and resources (like instances, security groups, floating IPs, Puppet groups, etc.) associated with a piece of software hosted on Cloud VPS. A project is a security concept. It's a group of users, a subset of which are given extra permissions as defined by the role: project member (formerly "projectadmin"). Cloud VPS (and Toolforge) projects are meant to reflect real-world endeavors, like "tools" or "bots".
  • An instance is just one resource that a Cloud VPS project might use. An instance is a virtual machine. Every instance belongs to a project. We are using EC2/OpenStack terminology here. When creating a new instance, the user can decide how much memory and storage space the virtual machine will have. See Help:Instances for more details.

Why create a Cloud VPS instance?

Cloud VPS is a flexible option for those who need it. It is equivalent to having your own server(s).

Before requesting a Cloud VPS instance, we recommend you consider using Toolforge instead. Toolforge has many of the same features as Cloud VPS, and Toolforge users do not have to manage the full virtual environment. Toolforge users also have access to toolsdb, wiki replicas, and a number of other services.

See Help:Cloud Services Introduction to help decide which service is right for you.

Who can create a Cloud VPS instance?

Project members (formerly called project admins). You must be a Project member in a project to create and manage its instances.

What are the responsibilities of project members?

Project members must subscribe to the cloud-announce and cloud mailing lists and follow the maintenance or communication actions requested by the WMCS Admins. Failure to respond may result in unexpected loss of data or service interruptions.

Project members should delete unused instances and notify other members when a project is no longer active. See Cloud VPS instance lifecycle for more details.

Requesting Project member rights

To create instances, individuals must have project member rights on the specific project they want to have instances created for. The following people can grant Project member rights:

  • Project members. Project members can grant project member rights on projects they belong to. You can find Project members listed on the individual pages for the projects.
  • Cloud VPS admins. Cloud admins, including Wikimedia Cloud Services staff and trusted community volunteers, can manage membership and Project member status in any project.

Contacting project members and admins
Project members and members of the Cloud admin group can be contacted through the 'email a user' function or through their talk page on Wikitech. Please note: Many members of the Wikitech community are volunteers who may not check their email or talk pages regularly. You may need to check with more than one person before you receive a response.

Project access rights
Learn more about Cloud services user roles and rights.

Before you create a Cloud VPS instance

  • Be a Project member in the project you intend to create an instance for.
  • Make sure your project has an appropriate security group for the instance's networking needs.
  • Be aware of your project's quotas (usage limits).
  • Understand the Cloud VPS instance lifecycle.

Create a Cloud VPS instance

Pre-requisites: Setup accounts and enable multi-factor authentication

  1. Head over to Help:Create a Wikimedia developer account to create a Developer Account.
  2. In your developer account enable Multi-Factor Authentication by taking the following steps:
  • First, click on Preferences.
  • Then click on Manage, listed under the Two Factor-authentication option.
Two Factor-authentication option
  • Select Enable TOTP (one-time token) method.
  • You can make use of the Google Authenticator App or the Authy app (which has some advantages like encrypted backups). If you download the Google Authenticator app on your phone and click on the “+” sign.
  • Select the option ‘Scan QR code’.
  • Fill out the unique code on the website.

Working with Cloud VPS Instances

  • Log in to https://horizon.wikimedia.org/.
  • Switch to the Project where you will host your VM. This option is available at the top-left of your Cloud VPC portal.
Instances section
  • The interface should change and you should be able to view, launch and delete Instances. This implies that you have project member privileges.The highlighted column at the top left of the image above, provides you an option to select a project scope you can provision your machine in. You can be in one or multiple projects within openstack.
Instances page when you have project member privileges

The steps below will guide you through setting up a VM on this interface:

  • Fill out the form as presented below. Ignore the Availability zone option and leave Count as 1. Then click Next.
Launch Instance - Step 1
  • Select an Operating System Image from the Available section.
Launch Instance - Step 2
  • Do this by clicking the up-arrow icon to move the image into the Allocated section, and click Next.
Launch Instance - Step 2b
  • Select the Flavor. The flavor influences how much RAM, CPU and disk your VM gets. You are limited to a few predetermined combinations here. Click Next.
Launch Instance - Step 3
  • Leave the Security Groups section as default. Click Next.
  • Leave Server Groups as default.
Launch Instance - Step 5
  • Clicking the Launch Instance button launches your new instance in the Spawning state, as seen in the instance overview.
Launch Instance - Step 6
  • Click on an instance’s name to view its details. The output should be similar to the image below:
Launch Instance - Step 7

To configure this instance with a puppet role for a specific job, click Puppet Configuration. You can apply puppet classes and, optionally, Hiera config values:

Launch Instance - Step 7b

Puppet Configuration for Cloud VPS instances

  • Cloud VPS uses Puppet to manage instance configurations.
  • If you are a sudo user for the project, you can run sudo puppet agent -tv or wait until puppet runs on your instance.

Logging into VMs with SSH

  1. Generate an SSH Key pair on your local machine
  2. Add the public key to your developer account in SSH key section on https://idm.wikimedia.org
  3. Then you can SSH into the instance

$ ssh -J <shell-name>@bastion.wmcloud.org <shell-name>@<instance>.<your-project>.eqiad1.wikimedia.cloud


shell-name is the name you picked when signing up for you developer account.

instance is the instance name you picked when creating the VM.

your-project is the name of the project you are building under

Note: Copy and paste only the alpha-numeric texts generated by your SSH agent in the text box provided on your developer account.

Instance information

Information type Action
Instance Details The name, zone, and count of your instance The name will also be your hostname. Name your instance using the following format project-instance, e.g, the project name is myproject; the instance is coolbot; the instance name is myproject-coolbot
Instance Source The version of the operating system (Debian Stretch, Debian Buster, etc). Also known as a base image.
Instance Flavor Contains a range of possible instance types indicating RAM, number of CPUs, and Hard Drive space. Your most likely choice will be at the top of the list of system images available. See current flavors in following chart. This decision is not permanent, you can allocate more resources later by using the red Action "Resize Instance".
Security Groups Contains a list of security groups At minimum, the default security group should be selected.*
Server Groups Contains a list of server groups Server groups allow you to define logical groups of instances with advanced scheduling features. Note: avoid using the affinity server group policy. See the linked help page for why.

* If the Project members for the project have not created any additional security groups, besides the default group, you will only see the default security group listed.

Instance Sizes and Flavors

Cloud VPS sizes*
g4.cores1.ram2.disk20 1 2 GB 20 GB Yes
g4.cores2.ram4.disk20 2 4 GB 20 GB Yes
g4.cores4.ram8.disk20 4 8 GB 20 GB Yes
g4.cores8.ram16.disk20 8 16 GB 20 GB Yes
g4.cores16.ram16.disk20 16 16 GB 20 GB Yes

* Current as of June 2024

Increase quotas for projects

Quotas are the usage limits for projects. Quotas refer to one or more of CPU, RAM, disk storage, number of VM instances and/or floating IPs. The default quota for a new project allows up to 8 instances.

Projects can have their quota increased following the instructions on the "Cloud-VPS (Quota-requests)" Phabricator task.

The Cloud VPS Instance lifecycle

Instances are allocated resources (storage, RAM, CPU, power). Over time, these resources need to be used or reclaimed. WMCS admins periodically check projects and instances to ensure they are being used by active projects. Instances will be removed for projects that have been determined inactive.

Process for removing Cloud VPS instances

  1. Notice will be sent to cloud via cloud-announce at least 2 weeks before any expected action is taken. Emergencies may require more immediate intervention.
  2. If possible, instances will be suspended or shut down for at least 2 weeks (a total of a month) before further unsolicited actions are taken to allow the tenant to notice.
  3. Inactive projects (those without instances or any responsive users) will be removed (along with their quota)

Backups of Cloud VPS instances

No backups will be kept by Wikimedia Cloud Services.

Local backups

You can backup your server by copying the files you are interested to save on your local machine. Of course, this is not an automated process and you will need to store the data you are saving. Also, be mindful that backups may be very big and so you will be transferring a lot of data, which among other things will take a long time.

If you want to backup your root disk on your VPS server to a local directory named backup you can do the following:

rsync -aXvz --stats --info=progress2 --rsync-path="sudo /usr/bin/rsync" --exclude-from=./excludes.txt <server>:/ backup/`

The paths listed in the file excludes.txt will be ignored. This gist contains a common list of excluded paths that you usually do not need to backup since they are temporary files, cache, backups and other system files.

Communication and support

We communicate and provide support through several primary channels. Please reach out with questions and to join the conversation.

Communicate with us
Connect Best for
Phabricator Workboard #Cloud-Services Task tracking and bug reporting
IRC Channel #wikimedia-cloud connect General discussion and support
Mailing List cloud@ Information about ongoing initiatives, general discussion and support
Announcement emails cloud-announce@ Information about critical changes (all messages mirrored to cloud@)
News wiki page News Information about major near-term plans
Blog Clouds & Unicorns Learning more details about some of our work

See also