Jump to content

Nova Resource:Openrefine/Documentation

From Wikitech

Openrefine

Description
OpenRefine is a tool to work with, clean up, and reconcile data sets, with special support for Wikidata. This service hosts a public installation of that tool, which can be used by users who are unable or unwilling to install it locally, or by groups of users who want to collaborate on a project.

Anticipated traffic level
100-1000 hits per day
Anticipated time span
indefinite
Project status
currently running
Contact address
phabricator:T194767
Willing to take contributors or not
willing

Technical setup

The project currently contains no instances. The previous openrefine01 instance was deleted in T238003; its documentation is preserved below.

Extended content

The project contains a single instance, openrefine01 (SSH: openrefine01.eqiad.wmflabs), which is based on a debian-9.3-stretch image. OpenRefine is installed under /opt/openrefine/ as a git clone (run git pull and then ./refine build to update). It is run from the openrefine.service systemd unit, with the data directory set to /srv/openrefine/workspace.

/srv is a separate volume, as per Help:Adding Disk Space#Adding space as a second volume. Most of its space is occupied by the file /srv/openrefine.raw, which is loop-mounted as /srv/openrefine using the srv-openrefine.mount systemd unit. It is a btrfs file system, and the workspace subvolume is periodically snapshotted (by the openrefine-workspace-snapshot.timer and openrefine-workspace-snapshot.service systemd units) under .snapshots, as a rudimentary form of backups.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Openrefine/Documentation&oldid=1844352"