Portal:Cloud VPS/Admin/Haproxy

From Wikitech
Jump to navigation Jump to search

This page contains information about our HAProxy configuration in the CloudVPS environment.

OpenStack API Endpoints

Backend HA Status

Backend services that are full active / active will automatically be pooled and depooled by HAProxy. As long as there is one instance available there will be no interruption in service.


Service active backends standby backends notes
Designate cloudservices1003
Glance API cloudcontrol1003 cloudcontrol1004 Waiting on shared storage (CEPH) for full active/active
Glance Registry cloudcontrol1003 cloudcontrol1004 Waiting on shared storage (CEPH) for full active/active
Keystone cloudcontrol1003
Neutron cloudcontrol1003
Nova cloudcontrol1003

Port Mappings

Service Backend Port Frontend Port
Designate API 9001 9001
Glance API 19292 9292
Glance Registry 19191 9191
Keystone Admin 15000 5000
Keystone Public 45357 35357
Neutron 19696 9696
Nova API 18774 8774
Nova Metadata 18775 8775

Load Balancing and Session Handling

Backend connections for each service are dynamically directed to the server with the least amount of connections using the `leastconn` load balancing algorithm.

Session persistence is maintained with HAProxy stick tables using the IP address of the client. This will ensure new client connections get forwarded to the same server every time.



Icinga is configured with the following HAProxy process checks:

  • nrpe_check!check_haproxy_alive!10
  • nrpe_check!check_haproxy!10


The prometheus haproxy exporter is collecting haproxy backend and frontend metrics directly from haproxy through the URL http://localhost:9900/stats;csv. This URL is read-only and only provides statistic information through the loopback interface.

Grafana dashboard: https://grafana.wikimedia.org/d/tanisM2Zz/wmcs-openstack-eqiad1-api-stats



HAProxy statistics and metrics can be queried through the "stats socket" at `/run/haproxy/haproxy.sock`.

Process information:

echo "show info" | sudo socat /run/haproxy/haproxy.sock stdio


echo "show stat" | sudo socat /run/haproxy/haproxy.sock stdio

Log files

  • /var/log/haproxy/haproxy.log

Primary Failover


Each controller is running HAProxy with no cross dependencies on other controllers or HAProxy instances. Each load balancer is operating on its own with no shared information like client sessions between them.


Each region has a dedicated FQDN (openstack.<region>.wikimediacloud.org) which maps to one of the OpenStack controllers running HAProxy. In the event of failure or maintenance this DNS entry will need to be remapped to the other controller in the operations/dns git repository at https://gerrit.wikimedia.org/r/admin/projects/operations/dns