Citoid

From Wikitech
Jump to navigation Jump to search

Citoid is a stateless node service on the SCB cluster. It processes HTTP requests containing a URL, ISBN number or other identifiers/search terms by retrieving citation information from external sources with help from a local nodejs Zotero server. It responds with a JSON structure containing the citation information.

Deployment

These are directions for deploying citoid, a nodejs service. More detailed but more general directions for nodejs services are at Migrating_from_scap-helm#Code_deployment/configuration_changes.

Locate build candidate

From gerrit, locate the candidate build. PipelineBot will post a message with the build name, i.e.

PipelineBot
Mar 15 9:22 PM

Patch Set 4:

Wikimedia Pipeline
Image Build SUCCESS

IMAGE:
 docker-registry.discovery.wmnet/wikimedia/mediawiki-services-citoid

TAGS:
 2019-03-15-000-production, 950e3b4468f2f84d3bb23c33ba68d8a26725

'2019-03-15-000-production' is the name of the build.

Add change via gerrit

  1. Clone deployment-charts repo.
  2. vi values.yaml
main_app:
  image: wikimedia/mediawiki-services-citoid
  limits:
    cpu: 10
    memory: 4Gi
  liveness_probe:
    tcpSocket:
      port: 1970
  port: 1970
  requests:
    cpu: 200m
    memory: 200Mi
  version: 2019-01-17-114541-candidate-change-me
  1. Make a CR to change the version value for all three servers (staging, codfw, and eqiad), and after a successful review, merge it.
  2. After merge, log into a deployment server, there is a cronjob (1 minute) that will update the /srv/deployment-charts directory with the contents from git.

Navigating the server

Ssh into the deploy machine.

ssh deploy1001.eqiad.wmnet

citoid runs on two of the available server farms, codfw and eqiad. You can confirm this by doing

Change into the citoid directory.

cd /srv/deployment-charts/helmfile.d/services

List the directories

> ls
codfw  eqiad  staging

These are the three available clusters. You must upgrade each one separately.

Staging

>cd staging/citoid
>myusername@deploy1001:/srv/deployment-charts/helmfile.d/services/staging/citoid$ 
>source .hfenv
>helmfile diff
>helmfile apply

Apply may take awhile.

>helm status staging (to check status again)

To confirm the staging cluster works, try a request:

curl -X GET --header 'Accept: application/json; charset=utf-8' 'staging.svc.eqiad.wmnet:1970/api?format=mediawiki&search=979%201029801297'

Production

>myusername@deploy1001:/srv/deployment-charts/helmfile.d/services/codfw/citoid$ 
>source .hfenv
>helmfile diff
>helmfile apply 
>helm status production

Verify citoid is running with a curl request:

curl -X GET --header 'Accept: application/json; charset=utf-8' 'citoid.svc.codfw.wmnet:1970/api?format=mediawiki&search=979%201029801297'

Repeat the same process for the codfw cluster. Then verify with a curl request:

curl -X GET --header 'Accept: application/json; charset=utf-8' 'citoid.svc.eqiad.wmnet:1970/api?format=mediawiki&search=9791029801297'

Testing behind restbase

And finally, you can test the restbase implementation which should give the same results@

https://en.wikipedia.org/api/rest_v1/#!/Citation/getCitation

Monitoring

  • Icinga: Search for 'citoid'

Logs

Kibana: https://logstash.wikimedia.org/app/kibana#/dashboard/default search for 'type:citoid'.

Use kibana to look at logs. However, if nessecary, you can look at the raw logs from the cluster directory,

source .hfenv 
kubectl logs -l app=citoid -c citoid-production

See also