Help:Using a web proxy to reach Cloud VPS servers from the internet
This page describes how to create a simple web proxy for a Cloud VPS instance.
Before you begin
Before you begin, you must:
- Be a Project admin for the project that contains the instance you wish to proxy.
- Set up security groups properly (see § Security groups below, for details)
Benefits
- Ensures that TLS encryption is used between the front proxy and the client.
- Saves an IPv4 address and instead uses a name-based vhost. IPv4 addresses are a precious commodity in the modern internet.
- Protects both the end user and your Cloud VPS project from leaking the client IP address which is considered personally identifiable information by the Wikimedia Foundation by hiding the client IP address from your service.
Limitations
- TLS encryption between the proxy and your upstream service is not currently supported. Requests from the frontend to your upstream will however include a X-Forwarded-Proto: https header that your software stack can use to determine that TLS encryption with the client has been handled already.
Creating a web proxy
- Go to https://horizon.wikimedia.org/
- Select your project.
- Click "Network" in the left navigation menu.
- Click "Web Proxies" inside the expanded "Network" section of the left navigation menu.
- Click the "Create Proxy" button in the upper right of the page.
- In the "Hostname" field, enter the hostname that you wish to have as the publicly-visible name for your instance. Important: Enter just the hostname (e.g. 'webtastic'), not the fully qualified name (e.g. 'webtastic.wmcloud.org').
- Specify the domain for your instance using the “Domain” selector. If you want a domain that is not already present in the menu, a cloud admin (most likely a staff member) will need to create it for you.
- Select the instance that you're creating a proxy for in the “Backend Instance” selector.
- Enter the “Backend port” that the proxy should connect to on your instance. This will probably be either the default value of
80if you are running a normal web server installed for the system package manager. If you're using some other web server, refer to its documentation for the port to use. - Click “Create proxy”.
An entry for the new web proxy should appear in the proxy table.
Note: Remember that if you did not do it yet, you might have to configure the security groups properly to let the external traffic reach your VM (see § Security groups below).
From the terminal
Web proxies are a Cloud VPS specific construct and therefore cannot be created with an OpenStack Command-line client. There is however an unofficial CLI written in Go to create to create Web Proxies programmatically, mkwebproxy.
Security groups
You may need to update your security group settings before you can access the instance on an external browser.
Some projects have setup a 'web' security group to make this easier or already added the common 80 and 8080 ports to their default security group.
- In the left-hand navigation, open "Network", then "Security Groups" section and select the project you want to make available externally.
- Add a new rule to open up your proxied port for outside access.
- Set the start and end port in the rule form to the value you entered for “Backend port” when creating the proxy.
- Select "tcp" as the protocol
- CIDR range: 172.16.0.0/21
- Click "Submit"
You may also have to apply this new (or existing) security group to the instance you want to make available: navigate to "Instances" (in the "Compute" section) and select "Edit Security Groups" from the "Actions" menu of the appropriate instance.
Delete unused web proxies
Once your instance is no longer in use, make sure to delete the unused web proxy.
If you want to keep the domain but redirect somewhere else (for example, your tool has moved from Cloud VPS to Toolforge), you can use the redirects project to handle the redirect.
Other features
Migrate from a *.wmflabs.org proxy to a *.wmcloud.org proxy
Since 2020-07-06, newly created proxies use the wmcloud.org domain by default instead of the legacy wmflabs.org domain. Projects which have been using a *.wmflabs.org proxy can migrate to a *.wmcloud.org proxy by following these steps:
- Create a new *.wmcloud.org proxy pointing to your backend service
- Test your service using the new hostname while the related *.wmflabs.org proxy still exists
- Once you are ready to redirect all traffic to the *.wmcloud.org hostname, just delete the legacy *.wmflabs.org proxy. The Cloud VPS HTTP proxy service will automatically issue a redirect from <host>.wmflabs.org to <host>.wmcloud.org when there is no existing proxy for <host>.wmflabs.org.
wmcloud.org zone delegations
If your project needs a particularly large number of domains, the Cloud VPS admins can configure an entire subdomain of wmcloud.org to be used by the web proxy in your project. This will allow using names such as some-service.your-project.wmcloud.org. To request such a configuration, please file a Phabricator task in the Cloud-VPS project.
A wildcard proxy (*) can also be configured in a delegated domain to send traffic with no specific backend to a default backend.
Administrator documentation for setting this up is at Portal:Cloud VPS/Admin/Web proxy#Enable per-project subdomain delegation.
Vanity domains
It is now technically possible to use your own domain for a web proxy. This requires manual configuration by the Cloud VPS admins, who generally require some kind of justification why a wmcloud.org subdomain can not be used instead. Such valid use cases might include (but are not limited to) projects being transferred from some external hosting to Cloud VPS as well as affiliates or other established organizations wishing to use their existing domain name. To request such a configuration, please file a Phabricator task in the Cloud-VPS project.
Once approved, you need to point your domain to the Cloud VPS proxy:
- Root domains
- If your DNS provider supports aliasing at the root domain (this might be called "CNAME flattening", "ALIAS", "ANAME", or something similar), you can use that to point to the
proxy.project-proxy.eqiad1.wmcloud.orgservice name. - Otherwise, you should add an A record pointing to the proxy's IPv4 address, which currently
185.15.56.49. IPv6 is not currently supported, although it is being worked on. - Subdomains
- Add a CNAME record pointing to
proxy.project-proxy.eqiad1.wmcloud.org.
Once the domain has been pointed to the proxy, notify the Cloud VPS admin handling your request and they will complete the remaining setup on the proxy side. Each individual domain name to be used will need to be configured individually; for example example.org and www.example.org are considered different names.
When configuring the web proxy, use @ in the "Hostname" field to use the domain specified in the "Domain" dropdown without an additional subdomain.
Note that each individual (sub)domain needs to be manually configured to have a valid TLS certificate issued. It is not currently possible to acquire wildcard certificates for domains with DNS hosting not handled by the Cloud VPS authoritative name servers. Those servers have not been previously been used to host custom domains, and any future such uses would require extensive discussion amongs Cloud VPS admins on whether that's a use case they want to support.
Administrator documentation for setting this up is at Portal:Cloud VPS/Admin/Web proxy#Enable support for a custom domain.
Troubleshooting
There are two reasons why web pages cannot be served by a default Cloud VPS instance:
- Instances are closed off from outside networks with a firewall. You must open holes in the firewall by editing the security groups for your project.
- Instances are assigned private IP addresses that are only visible from within Cloud VPS. This can be addressed by assigning your instance a public IP or by creating a web proxy.
Communication and support
Support and administration of the WMCS resources is provided by the Wikimedia Foundation Cloud Services team and Wikimedia movement volunteers. Please reach out with questions and join the conversation:
- Chat in real time in the IRC channel #wikimedia-cloud connect or the bridged Telegram group
- Discuss via email after you have subscribed to the cloud@ mailing list
- Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
- Read the News wiki page
Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself
Read the Cloud Services Blog (for the broader Wikimedia movement, see the Wikimedia Technical Blog)