Jump to content

Bits.wikimedia.org/Bits and pieces plan

From Wikitech
This page contains historical information. It may be outdated or unreliable.
2010

The idea of bits and pieces is moving serving a group of certain most viewed static objects (css, js, etc) via another set of cache servers - that would be configured for many many connections, many many requests and not too much data with nearly no invalidations.

Rationale

Currently we have quite heavy mix of our standard "needs huge cache" and small assets workload:

  • 18% of all squid load is for /skins*
  • 10% of all squid load is for /w/extensions/*
  • 3% of hits are for centralnotice (even after fundraiser)
  • 12% of hits are for action=raw css/js files

Frankly, that is most of our standard text squid load, so by moving it away we're making text cluster purely article-caching and mediawiki request routing. Though moving action=raw requests may not be feasible immediately, it is a possibility for future development. Initial target plan is using bits and pieces for /skins and /w/extensions.

This has already caused stability problems in past (e.g. new Usability Initiative assets have overloaded our systems), so we should do quite aggressive initial sizing for newly built platform (we can target for 100k-150k requests/s).

Software

  • Such deployment is ideal for initial Varnish testing in our environment. Of course, there may be need to fall back to Squid or Sun Java Enterprise Cache. ;-)
  • Alternative is having software deployed to nginx or lighttpd instances on the edge.
  • We may also consider switch-based load balancing here, as workload will be uniform

Deployment plan

  • Pick name ;-) bitsandpieces.wikimedia.org looks fine! Alternatively we can use 'bits.wikimedia.org' or 'pieces.wikimedia.org' ;-) If we want to be industry buzzword compliant it would be called 'assets'.
    • currently it is bits.wikimedia.org
  • Configure virtual host on all apaches, that would be configured to statically deliver /w/extensions and /skins/, and not serve mediawiki itself (403 on PHPs, no redirects, etc - would do fine)
    • done
  • point the name to text cluster RRs
    • done
  • Make sure Squid deals fine with this new domain and files.
    • done
  • Change inside MediaWiki to use new domain (URL structure can be same, just shared domain)
    • done for core and UsabilityInitiative/*
  • Set up new geo-balancing domain
    • done: bits-geo.wikimedia.org
  • Set up new VIP on load-balancers
  • Add new caches to new VIP
  • Start changing VIP for 'bits' to new-cache-based pool
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Bits.wikimedia.org/Bits_and_pieces_plan&oldid=817844"