LVS and Varnish

From Wikitech
Jump to: navigation, search
Wikimedia infrastructure

Data centres and PoPs

Networking

HTTP Caching


Wiki

Media

Logs

LVS and Varnish infrastructure

Wikipedia request flow

Diagram of "text" traffic flow through Wikimedia front edge LVS/nginx/Varnish infrastructure.

Notes

esams is representative of all cache-only sites; ulsfo is similar.

"upload" and "misc" traffic operate similarly to "text" shown here.

When LVS balances traffic to ports :80 (varnishes), and :443 (nginxes), it uses a hash of the client IP to help with TCP Fast Open and SSL session persistence respectively.

Within the caching layer (cp[1234]xxx machines), the jump from nginx to varnish:80 is direct on the local host only.

However, the jump from varnish:80 (frontend) to varnish:3128 (backend) is different: for that jump, we hash on the URL (etc) when balancing to the backends to divide the cache space among all machines, and thus the request typically moves from one machine to another within the cache layer here.