jobrunner is a service that continuously processes items off the MediaWiki job queue.
The job queue provides a means of deferring work that is too expensive to perform in the context of a web request. On our production environment, it does this by using Redis to enable shared access to a queue. Web MediaWiki instances enqueue operations for asynchronous execution, and a special class of app servers called job runners dequeue and execute them. The master process on the job runners is a service implemented in PHP called jobrunner.
dispatcher configuration option specifies how a batch of jobs will be run. By default this uses the runJobs.php maintenance script. For Wikimedia specifically, the dispatcher is configured to instead make an HTTP request to an RPC endpoint on the localhost (docroot:/rpc/RunJobs.php). This allows it to optimally use HHVM (command-line invocation would have a higher startup time and no persistent compilation cache).
When the jobrunner service starts, it reads configuration values from
/etc/jobrunner/jobrunner.conf. This file is generated by Puppet from puppet:/modules/mediawiki/manifests/jobrunner.pp and puppet:/modules/mediawiki/templates/jobrunner/jobrunner.conf.erb. For an overview of configuration options, see the jobrunner.sample.json file.
- Browse to the deployment directory and get the local repo in the state you want to deploy (e.g.
- Once ready, first run
scap deploy-login one terminal (or screen) to start watching the logs.
- In another terminal (or screen), run
scap deploy -v "log message here"to start the deployment.
- Follow the instructions as deployment reaches each group of servers. Scap will automatically restart services on active jobrunner servers (e.g. those in the primary DC).
To manually restart the jobrunner and jobchron services without a deployment:
tin$ cd /srv/deployment/jobrunner/jobrunner tin$ scap deploy -v --service-restart "restarting jobrunners in active dc"
This can be limited to specific hosts as follows:
tin$ cd /srv/deployment/jobrunner/jobrunner tin$ scap deploy -v --service-restart --limit-hosts [hostname or range] "log message"
You can push a null job to the jobqueue via
> JobQueueGroup::singleton()->push( new NullJob( Title::newMainPage(), array() ) );
Logging and metrics
The jobrunner service logs to