How to deploy Wikidata code

From Wikitech
Jump to: navigation, search

Code

Wikidata code is now deployed as a regular set of extensions.

THERE IS NO WIKIDATA BUILD. =]

Configuration changes

Wikidata configuration is in the operations/mediawiki-config repository.

The main files you need to edit are:

  • wmf-config/Wikibase.php: Wikidata specific configuration
  • wmf-config/InitialiseSettings.php: the options you want to enable for a specific wiki
  • wmf-config/CommonSettings.php: the logic matching IS options and their effect

You've basically two solutions to deploy your change:

  • Normally aude, hoo or addshore handle these
  • You can also sign up for a SWAT deploy where someone will deploy for you, point them to this page if they have Wikidata specific questions.

There are no deploys on Fridays (and weekends) except for emergencies. See below for the procedure to follow in this case, basically ping Greg or if missing no_justification on Freenode #wikimedia-operations.

Schema changes

Schema changes should be reviewed by the WMF's database engineer. There are database specifics procedure and notes available at MediaWiki Database policy and at Schema changes.

In a nutshell, you need to know:

  • database schema change won't be applied as the same to the code
  • for any schema change, you need to create a task on Phabricator, tagged #DBA

After the schema change is done:

  • Add table to dumps: Add the table to the dump python scripts (https://phabricator.wikimedia.org/diffusion/ODUM/), by either poking Ariel (apergos) or making a patch yourself + poke Ariel. The 'ariel' branch of the dumps git repo is currently (as of July 2014) being used, so make the patch there. See 71087 for an example. A good description of the table is important. If the table contains any private information (e.g. ip addresses), then adding it to the dumps is more complicated and it would be best for Ariel to do it.
  • Add table to toollabs: The scripts for managing database replications on labs are in https://phabricator.wikimedia.org/diffusion/OSOF/. An example of how to add tables is https://gerrit.wikimedia.org/r/#/c/118582/ and how to have a column added is https://gerrit.wikimedia.org/r/#/c/125369/. Then poke DBAs and Labs team to review / deploy. If the table involves private information (e.g. ip addresses), then it is best just to poke Coren and Sean and let them handle it.

Update cron jobs

Dispatch changes and some other stuff is run as a cron job. Cron jobs are configured in puppet, in "manifests/misc/maintenance.pp".

https://phabricator.wikimedia.org/diffusion/OPUP/browse/master/manifests/misc/maintenance.pp;55e76106e0e976c5d56ccb2a54921ad90bbe764d

Any cron jobs that you want to stop need to be marked as "absent" and do not get removed from puppet manifest. That allows the servers to know this job must be removed where it exists.

To change a cron job:

  • if you need to edit the cron parameters, add a new cron job and mark the old one as "absent"
  • if you need to edit the script called by the cron job, prepare a change with this change, but the current cron job can be kept

How to make a Wikidata build for master / beta deployment

Builds for beta are generated daily on a labs instance, which is in the Wikidata-build project. Both WikidataJenkins and wmf-jenkins will run tests on any new builds. Once they both approve, then someone needs to +2 code review and then Jenkins will merge the new build. Builds for master branch of Wikidata.git will be deployed automatically to beta.

See also