Jump to content

Help:Using OpenTofu on Cloud VPS

From Wikitech

This page contains instructions and best practices for using OpenTofu, a popular open-source infrastructure as code tool, to manage Cloud VPS resources.

OpenTofu is a fork of Terraform. Not everything on this page has yet been updated to match the fork and name change.

Demo

See the asciinema recording for a demo of attaching a volume to an existing OpenTofu-managed instance.

OpenTofu documentation

To get familiar with OpenTofu itself, refer to the official documentation.

Setting up OpenTofu to work with Cloud VPS

Provider

Since Cloud VPS uses OpenStack, the standard Terraform OpenStack provider can be used to manage some resources. Note that not all OpenStack features are available on Cloud VPS.

There is also a custom Cloud VPS provider for managing Cloud VPS specific features. It can be installed with setting the provider source attribute to terraform.wmcloud.org/registry/cloudvps. Automatically generated documentation for the provider is available on doc.wikimedia.org.

Authentication

For security reasons, direct access to the Cloud VPS APIs using a developer account username and password is disabled. Instead, you must create an application credential to work with the APIs.

State management

Tracked in Phabricator
Task T318360

You need to configure an OpenTofu state backend if you want to manage the same resources from multiple different machines. The Cloud VPS object storage service has been tried and works with the S3 interface. The configuration required is like this: 

terraform {
  backend "s3" {
    region   = "eqiad1"
    bucket   = "[PROJECT_ID]:[BUCKET]"
    endpoint = "https://object.eqiad1.wikimediacloud.org"
    key      = "state/main"

    skip_region_validation      = true
    skip_credentials_validation = true
    force_path_style            = true
  }
}

This depends on AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables for authentication.

If your Cloud VPS project is using the newer naming style where the Project ID is a UUID rather than a descriptive name, you must be sure to use the UUID value rather than the name.

Example setup

First, create a file called secrets.auto.tfvars and add your project name and application credential details: TODO: something about secret storage best practices here? 

os_auth_url                      = "https://openstack.eqiad1.wikimediacloud.org:25000"
os_project_id                    = "[replace me]"
os_application_credential_id     = "[replace me]"
os_application_credential_secret = "[replace me]"

Then, you can install and configure the required providers in your main.tf file: 

terraform {
  required_version = ">= 1.4.0"
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = "~> 1.51.1"
    }

    cloudvps = {
      source  = "terraform.wmcloud.org/registry/cloudvps"
      version = "~> 0.2.0"
    }
  }
}

variable "os_auth_url" { type = string }
variable "os_project_id" { type = string }
variable "os_application_credential_id" { type = string }
variable "os_application_credential_secret" {
  type      = string
  sensitive = true
}

provider "openstack" {
  auth_url                      = var.os_auth_url
  tenant_id                     = var.os_project_id
  application_credential_id     = var.os_application_credential_id
  application_credential_secret = var.os_application_credential_secret
}

provider "cloudvps" {
  os_auth_url                      = var.os_auth_url
  os_project_id                    = var.os_project_id
  os_application_credential_id     = var.os_application_credential_id
  os_application_credential_secret = var.os_application_credential_secret
}

Cloud VPS specific requirements

Referencing admin managed resources

You can use OpenTofu's data providers to reference objects managed by Cloud VPS admins, for example instance flavors, base images or the lan-flat-cloudinstances2b network object: 

data "openstack_compute_flavor_v2" "vm_flavor" {
  name = "g3.cores1.ram2.disk20"
}

data "openstack_networking_network_v2" "lan_flat_cloudinstances2b" {
  name = "lan-flat-cloudinstances2b"
}

data "openstack_networking_secgroup_v2" "default" {
  name = "default"
}

resource "openstack_compute_instance_v2" "demo_vm" {
  name      = "demo-vm"
  image_id  = "9a01d3d8-e793-4775-8b81-434f68c687a7" # see below about image ids
  flavor_id = data.openstack_compute_flavor_v2.vm_flavor.id

  security_groups = [
    # Remember to include the default security group, otherwise you can't SSH in and WMCS monitoring will think the instance is down
    data.openstack_networking_secgroup_v2.default.name,
  ]

  network {
    uuid = data.openstack_networking_network_v2.lan_flat_cloudinstances2b.id
  }
}

Base images

The available Debian version specific base images are refreshed from time to time, but you don't need to replace VMs once a newer image for the same Debian version comes out as you can just upgrade the packages on the instance itself. For this, you can use lifecycle configuration to ignore changes to the image after creation. For example: 

variable "image_name" {
  type    = string
  default = "debian-12.0-bookworm"
}

data "openstack_images_image_v2" "bookworm" {
  most_recent = true
  name        = var.image_name
}

resource "openstack_compute_instance_v2" "demo_vm" {
  # other required properties hidden for brevity
  image_id  = data.openstack_images_image_v2.bookworm.id
  
  lifecycle {
    ignore_changes = [ image_id ]
  }
}

In the example above, the data block always gets the latest image with the defined name, but OpenTofu is instructed to ignore any changes to the instance's configured image, and thus if the data block starts returning a different image (either due to you changing the value of the variable, or due to WMCS replacing the image), OpenTofu will just ignore that change entirely and leave the instance as-is.

To force OpenTofu to upgrade the instance to the latest image, you can run tofu apply --replace=openstack_compute_instance_v2.demo_vm - beware that this will destroy and recreate the instance.

Examples

Example OpenTofu setups

Communication and support

OpenTofu is not currently officially supported by the Cloud VPS administration team as a first-class management tool. This page and related tooling (such as the Cloud VPS OpenTofu provider) are maintained by community volunteers, some of which also have administrative access to the Cloud VPS platform itself. If you need help, you can still use the cloud mailing list and related channels.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Help:Using_OpenTofu_on_Cloud_VPS&oldid=2214567"