Tool:Prompter

Toolforge tools

Prompter
Website	prompter
Description	Generates prompts for users to update Wikidata by comparing external CSVs to what's in Wikidata
Keywords	csv, prompts, wikidata
Author(s)	EveryPolitician Team
Maintainer(s)	(View all)
Source code	https://github.com/everypolitician/prompter
License	MIT License

{"name":"prompter","repository":"https://github.com/everypolitician/prompter","author":"EveryPolitician Team","url":"prompter","keywords":"csv, prompts, wikidata","description":"Generates prompts for users to update Wikidata by comparing external CSVs to what's in Wikidata"}

Controlling the service

Start

become prompter
webservice --backend=kubernetes ruby2 start /data/project/prompter/prompter/script/toolforge-start

Stop

become prompter
webservice --backend=kubernetes ruby2 stop

Update code and restart

Toolforge doesn't currently, at least as far as I understand, offer zero-downtime deploys, so there will be a short time while deploying where the webapp will return a 502 error. This should resolve itself once the Kubernetes pod has started up correctly.

become prompter
cd prompter
git pull
webservice --backend=kubernetes ruby2 restart

Sometimes when running that last restart command you'll see a Python stacktrace from the webservice tool. This appears to be harmless, but it's prudent to check that the service has correctly by checking the running pods and viewing the logs for the service using the Kubernetes commands below.

Kubernetes commands

Check which pods are running

kubectl get pods

View logs for a pod

First copy the pod name from the output of kubectl get pods and then run:

kubectl logs -f <POD-NAME>

Refreshing prompts regularly with cron

In order to install the crontab I ran the following commands:

become prompter
crontab -e

Then I added the following crontab entry, which refreshes all prompts at 09:00 UTC:

0 9 * * *  /usr/bin/jsub -N cron-tools.prompter-1 -once -quiet /data/project/prompter/prompter/script/toolforge-cronjob

That runs ./script/toolforge-cronjob from the prompter repo, which in turn runs ./script/update-all-prompts.

Initial setup

This project uses Ruby 2.4.1, but the latest version available on Toolforge at the time of writing (17:36, 23 August 2017 (UTC)) is 2.1.5p273. So to get this running on Toolforge I first had to download and install a newer version of Ruby. I used Travis CI's Precompiled Ruby Versions to download a prebuilt version of Ruby 2.4.1. These are the exact steps I took:

become prompter
curl -sL 'https://rubies.travis-ci.org/ubuntu/14.04/x86_64/ruby-2.4.1.tar.bz2' | tar xjvf -
echo 'export PATH="$HOME/ruby-2.4.1/bin:$PATH"' >> ~/.profile
source ~/.profile
gem install bundler

Then to install the application code I did the following:

git clone https://github.com/everypolitician/prompter
cd prompter
bundle install --deployment --without development test --path vendor -j 3

With those steps complete I was able to start the service.

Other notes

• I had to make sure the app was listening on 0.0.0.0, not localhost, to get it running correctly in Kubernetes
• Use the PORT environment variable to determine which port to listen on.
• The ./script/toolserver-start script encapsulates starting the service, this is the script passed as the SCRIPT parameter to the webservice command.