WMDE/Wikidata/Deployment

From Wikitech
< WMDE‎ | Wikidata(Redirected from How to deploy Wikidata code)
Jump to navigation Jump to search

How to determine the deployed Wikidata version

Look on https://wikidata.org/wiki/Special:Version or https://tools.wmflabs.org/versions/

How to deploy code

Wikidata code is now deployed as a regular set of extensions. See How_to_deploy_code.

Configuration changes

Wikidata configuration is in the operations/mediawiki-config repository.

The main files you need to edit are:

  • wmf-config/Wikibase.php: Wikibase specific configuration
  • wmf-config/InitialiseSettings.php: the options you want to enable for a specific wiki
  • wmf-config/CommonSettings.php: the logic matching IS options and their effect

You've basically two solutions to deploy your change:

  • Normally hoo, amir or addshore handle these
  • You can also sign up for a SWAT deploy where someone will deploy for you, point them to this page if they have Wikidata specific questions.

There are no deploys on Fridays (and weekends) except for emergencies. See below for the procedure to follow in this case, basically ping Greg or if missing no_justification on Freenode #wikimedia-operations.

Schema changes

Schema changes should be reviewed by the WMF's DBAs. There are database specifics procedure and notes available at MediaWiki Database policy and at Schema changes.

In a nutshell, you need to know:

  • database schema change won't be applied as the same to the code
  • for any schema change, you need to create a task on Phabricator, tagged #DBA

After the schema change is done:

  • Add table to dumps: Add the table to the dump python scripts (https://phabricator.wikimedia.org/diffusion/ODUM/), by either poking Ariel (apergos) or making a patch yourself + poke Ariel. The 'ariel' branch of the dumps git repo is currently (as of July 2014) being used, so make the patch there. See 71087 for an example. A good description of the table is important. If the table contains any private information (e.g. ip addresses), then adding it to the dumps is more complicated and it would be best for Ariel to do it.
  • Add table to toollabs: The scripts for managing database replications on labs are in https://phabricator.wikimedia.org/diffusion/OSOF/. An example of how to add tables is https://gerrit.wikimedia.org/r/#/c/118582/ and how to have a column added is https://gerrit.wikimedia.org/r/#/c/125369/. Then poke DBAs and Labs team to review / deploy. If the table involves private information (e.g. ip addresses), then it is best just to poke Coren and Sean and let them handle it.

Update cron jobs

Dispatch changes and some other stuff is run as a cron job. Cron jobs are configured in puppet, in "manifests/misc/maintenance.pp".

https://phabricator.wikimedia.org/diffusion/OPUP/browse/master/manifests/misc/maintenance.pp;55e76106e0e976c5d56ccb2a54921ad90bbe764d

Any cron jobs that you want to stop need to be marked as "absent" and do not get removed from puppet manifest. That allows the servers to know this job must be removed where it exists.

To change a cron job:

  • if you need to edit the cron parameters, add a new cron job and mark the old one as "absent"
  • if you need to edit the script called by the cron job, prepare a change with this change, but the current cron job can be kept

Adding a client

If a client is being added as part of a new wiki, see Add a wiki#Wikidata.

Otherwise, first create the tables:

ladsgroup@mwmaint1002:/srv/mediawiki/php-1.33.0-wmf.1$ mwscript sql.php --wiki=sourceswiki extensions/Wikibase/client/sql/entity_usage.sql

Then, add name of the wiki to wikidatacleint.dblist and deploy it. For example: https://gerrit.wikimedia.org/r/#/c/472008/

You can check arbitrary access with something like the following: {{#property:P31|from=Q4115189}}

In order for dispatching to work the wiki needs to be listed in the wb_changes_dispatch table on wikidatawiki (it might already be there).

You can check the dispatch logs for the wiki by doing something like this:

addshore@mwmaint1002:/var/log/wikidata$ tail -f dispatchChanges-wikidatawiki.log | grep 'sourceswiki'

Using the example arbitrary access above making a change to P31 on Q4115189 should result in a log message in the dispatch log for the wiki after a ~30 seconds (depending on the health of the dispatcher)