Citoid

From Wikitech
Jump to navigation Jump to search

Citoid is a stateless node service on the SCB cluster. It processes HTTP requests containing a URL, ISBN number or other identifiers/search terms by retrieving citation information from external sources with help from a local nodejs Zotero server. It responds with a JSON structure containing the citation information.

Deployment

Deploying citoid involves deploying both the citoid service itself and the zotero service it's dependant on. For deploying the Zotero service, see Zotero/Deploying zotero

These are directions for deploying citoid.

Navigating the server

Ssh into the deploy machine.

ssh deploy1001.eqiad.wmnet

citoid runs on two of the available server farms, codfw and eqiad. You can confirm this by doing

scap-helm citoid list

It should list both clusters with deployment status.

Change into the citoid directory.

cd /srv/scap-helm/citoid

List the files

> ls
citoid-codfw-values.yaml  citoid-eqiad-values.yaml  citoid-staging-values.yaml

These files must be changed in order to update either production cluster or the staging cluster. It's a good idea to try the staging one first.

Locate build candidate

From gerrit, locate the candidate build. PipelineBot will post a message with the build name, i.e.

PipelineBot
Mar 15 9:22 PM

Patch Set 4:

Wikimedia Pipeline
Image Build SUCCESS

IMAGE:
 docker-registry.discovery.wmnet/wikimedia/mediawiki-services-citoid

TAGS:
 2019-03-15-000-production, 950e3b4468f2f84d3bb23c33ba68d8a26725

'2019-03-15-000-production' is the name of the build.

Staging

vi citoid-staging-values.yaml

Change the value of main app: version to the candidate build/

After saving, try upgrading the staging env.

CLUSTER=staging scap-helm citoid upgrade staging -f citoid-staging-values.yaml stable/citoid

It should confirm it works, but you can request: the status

CLUSTER=staging scap-helm citoid status staging

To confirm the staging cluster works, try a request:

curl -X GET --header 'Accept: application/json; charset=utf-8' 'kubestage1001.eqiad.wmnet:1970/api?format=mediawiki&search=9791029801297'

or

curl -X GET --header 'Accept: application/json; charset=utf-8' 'kubestage1001.eqiad.wmnet:1970/api?format=mediawiki&search=http%3A%2F%2Frspb.royalsocietypublishing.org%2Fcontent%2F267%2F1453%2F1627'

Production

To upgrade eqiad, do

 vi citoid-eqiad-values.yaml

Then to deploy, do:

 CLUSTER=eqiad scap-helm  citoid upgrade production -f citoid-eqiad-values.yaml stable/citoid

scap-helm citoid status production

To observe its status.

Verify citoid is running with a curl request:

curl -X GET --header 'Accept: application/json; charset=utf-8' 'citoid.svc.eqiad.wmnet:1970/api?format=mediawiki&search=9791029801297'

Repeat the same process for the codfw cluster:

vi citoid-codfw-values.yaml
CLUSTER=codfw scap-helm  citoid upgrade production -f citoid-codfw-values.yaml stable/citoid

CLUSTER=codfw scap-helm citoid status production

curl -X GET --header 'Accept: application/json; charset=utf-8' 'citoid.svc.codfw.wmnet:1970/api?format=mediawiki&search=9791029801297'

Testing behind restbase

And finally, you can test the restbase implementation which should give the same results@

https://en.wikipedia.org/api/rest_v1/#!/Citation/getCitation

Rolling back

CLUSTER=eqiad scap-helm citoid history production

to view previous revisions

CLUSTER=eqiad scap-helm citoid rollback production <revision-number-here>

To rollback the rollback show the history again:

CLUSTER=eqiad scap-helm citoid history production

And rollback to the desired revision.

Repeat

Repeat for codfw

Monitoring

Testing

These are example queries you can use to verify whether citoid basically works:

This request will query zotero:

curl 'http://localhost:1970/api?format=mediawiki&search=http:%2F%2Flink.springer.com%2Fchapter%2F10.1007%2F11926078_68'

This request will let citoid make the request itself:

curl 'http://localhost:1970/api?format=mediawiki&search=http:%2F%2Fwww.pbs.org%2Fnewshour%2Fmaking-sense%2Fcare-peoples-kids%2F'

Troubleshooting

By default, the service listens on port 1970. Example request:

curl -XPOST -d'format=mediawiki&url=http://link.springer.com/chapter/10.1007/11926078_68' http://sca1001:1970/url

Logs

/srv/log/citoid/main.log on each node.

Fail-over

The SCB cluster is load balanced through LVS. The failure of an individual node does not affect the availability of the service.

Restarting a node

service citoid restart

See also