MobileFrontend
Ops Configuration / Deployed Files
Varnish 3.0
- varnish3 class in varnish.pp (once bits is migrated, this will just be the varnish class)
- templates/varnish/mobile-frontend.inc.vcl.erb (port 80)
- templates/varnish/mobile-backend.inc.vcl.erb (port 81)
- hosts are in the mobile dsh group
MediaWiki
InitialiseSettings.php: wmgMobileFrontend - default => true
Traffic Flow
Redirect to mobile
Redirection from main wikis to their mobile version is handled by the frontend text Varnishes. Mobile devices are detected based on useragent and then redirected to an appropriate .m. domain if the "stopMobileRedirect" cookie is not set.
The logic is in the file templates/varnish/text-frontend.inc.vcl.erb, function mobile_redirect.
Mobile requests
The m.wikipedia.org vip is load balanced via lvs to a set of varnish servers running a set of front and backend varnish instances. The frontend varnish on port 80 includes VCL (puppet/templates/varnish/mobile-frontend.vcl.erb) that rewrites the host to drop the /m./ portion and sets an X-Subdomain header with values depending on Wikipedia Zero detection. The presence of this header activates the MediaWiki MobileFrontend extension and is used to determine which mobile template to use. The frontend instance uses 256MB of ram for caching and hashes requests to one of the backend varnish instances which provide larger caches using the file store.
Requests not cached in either varnish instance are sent to the main apache appserver vip. Responses are returned with the same X-Subdomain header added on in the frontend, and with X-Subdomain added to Vary.
Cache purging
Cache purging is handled via varnishhtcpd (installed and started via puppet - see varnish.pp) which listens to the multicast PURGE messages also used by the squids. It currently only sends http purge requests to the backend varnish. Since host names are rewritten to strip out /m./ at the frontend, the same PURGE messages just work. That said, varnishhtcpd uses the LWP::UserAgent perl module to send purge requests to varnish in http proxy mode, which results in requests that look like:
PURGE http://en.wikipedia.org/wiki/Cat HTTP/1.1 Host: en.wikipedia.org
This would never actually match a cached article, so there is code in vcl_recv() to strip out the (http://[\w.]+)/ portion.
To prevent having to send purge messages to the frontend as well (possibly requiring a host transform), the backend returns cache-control headers that set a ttl of 300 seconds for all pages. This could be a problem if we add mobile editing, although in that cases, logged in editing mobile users would likely bypass caching entirely.
Flushing the cache
Unlike the rest of our environment, it is OK to completely flush the mobile cache. The back end servers can handle it.
Running:
varnishadm ban.url . varnishadm -n frontend ban.url .
on each varnish server has the effect of flushing the entire cache. Despite the scary name, ban is a timed based action that means not to serve anything matching the pattern before the most recent matching ban. Varnish will still serve and recache everything.
To flush everything, check the dsh group is correct then blast out the flush commands.
- compare
http://noc.wikimedia.org/pybal/eqiad/mobileto/etc/dsh/group/mobileand make sure membership matches - on fenari, run:
dsh -g mobile "varnishadm ban.url . ; varnishadm -n frontend ban.url ."
Logging
Our varnish3 package contains a patched version of varnishncsa (/trunk/debs/varnish3/debian/patches/01-varnishncsa-udp.dpatch) that adds sequence numbers and udp logging with multicast support. The default format has been changed to be compatible with that of our squids. It is run as a daemon, reads from the shared memory belonging to the frontend instance (specified via the "-n frontend" option) and sends udp log messages to our existing logging infrastructure.
Deployment considerations/gotchas
MobileFrontend has some additional deployment procedures and gotchas not documented in other deployment docs. They are documented on MobileFrontend/Deployment.