From Wikitech
Jump to: navigation, search
Toolforge tools
Crystal Clear app package utilities.png XTools
XTools logo.svg
Description Suite of tools to analyze user and page data on WMF wikis
Keywords xtools, statistics, analytics
Author(s) Matthewrbowker, MusikAnimal, Samwilson
adapted from older version by X! and Hedonil
Maintainer(s) Matthewrbowker, Community Tech (View all)
Source code GitHub (Mirrored on Phabricator as rXTR)
License GNU General Public License 3.0 or later
Issues Open tasks · Report a bug
Admin log Nova Resource:Tools.xtools/SAL
Nova Resource:Tools.xtools-dev/SAL

This page is for documentation relating to the WMF installations of XTools at (production) and (staging). For general installation, configuration, and development of the XTools software, please see

There are two instances currently configured, one for staging and one for production (view details in the Openstack browser). The two instances relate to two Toolforge accounts: xtools and xtools-dev; these are where the matching database users come from, and where we send maintainers' emails.[citation needed]

Note to maintainers: we don't backup any server configuration, so please document everything here (until Task T170514 is resolved).


The maintainers can be emailed at tools.xtoolsAt (note that this means that the maintainers of three separate things need to be kept in sync: the VPS account and the two Toolforge accounts).


Production XTools is hosted on on a Wikimedia VPS instance. To log into the server, make sure you've been added as a maintainer to the xtools project. Then set up SSH and connect via ssh xtools-prod01.xtools.eqiad.wmflabs and go to the /var/www directory. Not quite everything in this directory is in the Git repository.

Database is s51187__xtools_prod on tools.labsdb, we're connecting as user s51187 (which is the same as the old XTools database user).

Web server configuration is all in /etc/apache2/sites-available/xtools.conf:

<VirtualHost *:80>
        DocumentRoot /var/www/web

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        <Directory /var/www/web/>
             Options Indexes FollowSymLinks
             AllowOverride All
             Require all granted

        Alias /awstatsclasses "/usr/share/awstats/lib/"
        Alias /awstats-icon/ "/usr/share/awstats/icon/"
        Alias /awstatscss "/usr/share/doc/awstats/examples/css"
        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        ScriptAlias /awstats/ /usr/lib/cgi-bin/
        <Directory /usr/lib/cgi-bin/>
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Require all granted


There's a /var/www/ script that runs every 10 minutes (from www-data's crontab) and if required updates to the latest release. The output of this is mailed to the maintainers.


cd /var/www

git fetch --quiet origin 2>&1

## Find the highest and current tags
HIGHEST_TAG=$(git tag | sort --version-sort | tail --lines 1)
CURRENT_TAG=$(git describe --tags)

## Exit and say nothing if we're already at the highest tag.
if [[ $CURRENT_TAG == $HIGHEST_TAG ]]; then
        exit 0

## If there's an update, pull and install it.
git checkout $HIGHEST_TAG
export SYMFONY_ENV=prod
/usr/local/bin/composer install --no-dev --optimize-autoloader
./bin/console cache:clear --env prod
./bin/console doctrine:migrations:migrate --env prod --no-interaction
./bin/console assetic:dump --env prod

Logs are written to /var/www/var/logs/prod.log, but only during a request where an error or high-priority log entry was made. This is why you'll see debug-level log entries in prod.log.

Web access stats are available at

OAuth consumer: XTools 1.0


SSH to xtools-dev02.xtools.eqiad.wmflabs (see notes above in #Production about getting access).

Database is s53003__xtools_dev on tools.labsdb, we're connecting as user s53003.

OAuth consumer: xtools-dev 1.0

The code on the staging server is kept up to date with the master branch with the following /var/www/ script (run every 10 minutes from www-data's crontab):


export SYMFONY_ENV=prod
cd /var/www

## See if there's any update.
GITFETCH=$(git fetch 2>&1)
if [ -z "$GITFETCH" ]; then
        exit 0

## If there's an update, pull and install it.
git checkout master
git pull origin master
/usr/local/bin/composer install --no-dev --optimize-autoloader
./bin/console cache:clear --env prod
./bin/console doctrine:migrations:migrate --env prod --no-interaction
./bin/console assetic:dump --env prod

Killing slow queries

Both servers run slow-query killing scripts for their respective database users.

Slow queries are killed with pt-kill by this daemon (see /var/www/

/usr/local/bin/pt-kill --user=xxxx --password=xxxx --host=enwiki.labsdb \
       --busy-time=90 \
       --log /var/www/web/killed_slow_queries.txt \
       --match-info "^(select|SELECT|Select)" \
       --kill --print --daemonize --verbose

The daemon can be stopped at any time by creating /tmp/pt-kill-sentinel.

The killed queries are publicly logged for both production and staging. These logs are kept to a maximum of 2 1 kb files via /etc/logrotate.d/xtools:

/var/www/web/killed_slow_queries.txt {
	size 1k
	rotate 2

Note also that pt-kill is installed manually (it's just a single Perl file), and that it requires libdbi-perl and libdbd-mysql-perl.

Installation on Toolforge

Old XTools is still installed on Toolforge at and new XTools was previously installed at — the instructions here are for the latter.

To create a XTools project on Tool Labs:

  • git clone
  • composer install
  • You will be asked a series of questions. See for documentation. For Tool Labs:
    • wiki_url – leave blank
    • app.is_labs = 1
    • app.single_wiki = 0
    • app.load_stylesheets_from_cdn = 0
    • app.replag_threshold = 30
    • Enable the tool that you are setting up, and set the rest to 0
  • You can use the app/config/table_map.yml file to force the app to use certain tables in place of the defaults. For instance on Tool Labs, you'll want to map revision to revision_userindex since it will be faster.
  • Set the .lighttpd.conf file to:
    url.rewrite-if-not-file += (
      "(.*)" => "/xtools-dev/app.php/$0",
      "^/$" => "$0"
  • Start the webservice with webservice --backend=kubernetes start

Proposed Labs setup

This is a proposed guideline - it's still under discussion

Once the new XTools enters beta, we must stabilize how we deploy the code. Unlike the old XTools, everything done on the rebirth must hit Git first. This allows for Travis, VersionEye, and CodeCov to run before the code is deployed.

We should own both subdomains "xtools-dev" and "xtools." Both serve a specific purpose. "xtools-dev" should pull from the latest master (This requires "master" to always be stable). xtools should run the latest stable tagged version of the software.

See also