Nova Resource:Deployment-prep/Roadmap

This is the roadmap for the beta cluster.

The roadmap is maintained from time to time following checkin meetings.

Each entry should have a corresponding bug in bugzilla "Wikimedia Labs" > "deployment-prep (beta)".

being worked on

• bug 49779 [Ariel] - VisualEditor corpus of articles for parser round trips and Selenium tests.
• restructure media storage setup to be more like production [Ariel]

In queue

• HIGH: setup monitoring for beta components (apacke/db/disk..) much like production

• HIGH import articles from production (based on dumps?)
  • Gadgets / CSS / Javascript
    • There is already a script copying some CSS from enwiki to enwiki.beta. Script in puppet*
    • Use case: let Twinkle developers try out their gadget against latest master of mediawiki.
    • Auto importing would be overriding possible changes made on the wikis.
  • Possible disk space issues when importing lot of articles on a daily basis.
  • Import should NOT be done with Special:Import or maintenance/ImportDump.php. Not reliable.
  • Should use one of the 3 alternatives: mwdumper (java), mwimport (perl), mwxml2sql (c)
  • Importer might break on MediaWiki DB schema change.

• clean up unused instances
• Elimination of shared storage

Future issues

• MariaDB
  • As of September 2013: Lucid + MySQL. We have two masters

1) deployment-sql.pmtpa.wmflabs : almost all wikis 2) deployment-sql02.pmtpa.wmflabs wikivoyage + maybe a few more

• Graphite would be super nice. Reach out to Asher?

• Different versions of extensions
  • As of September 2013: tip of master only.
  • Discussion among mw core team: people should use feature flags.

• Make sure all uids across beta instances are consistent
• public IPs of instances not reachable from other instances (generally for labs) bug 37985 bug 45868
  • This is caused by NAT, a workaround is to use an iptables rules on the instance to change the destination address from the public IP address to the private IP address of the instance. hashar (talk)
• Reuse the same apaches configuration files as in production

Completed

• Phase out the squid based text cache Done.
  • Mark made the varnish text manifest to be realm agnostic, been trivial to boot an instance using it and then switch all traffic to that new varnish instance hashar (talk) 19:38, 1 July 2013 (UTC)
• make purging work [Antoine]
  • Mostly working by using wgHTCPMulticastRouting. Still have to purge mobile cache hashar (talk) 19:39, 1 July 2013 (UTC)
  • Completed end of July 2013. Antoine "hashar" Musso (talk) 18:09, 4 September 2013 (UTC)
• search: deployed on beta by Chad + Nik with ElasticSearch and mw ext CirrusSearch. CirrusSearch is the extention that implements ElasticSearch on MW: https://www.mediawiki.org/wiki/Extension:CirrusSearch