Metrics Platform/MPIC/Administration

Metrics Platform Instrument Configuration (MPIC) is an instrument and experiment management system. The application enables product teams to set up, manage, and configure various instruments for data collection and experimentation. This includes scheduling, designating metadata, setting sample rates, and targeting deployment environments.

Service details

mpic-next (staging)

Attribute	Value
Owner	Data Products
Kubernetes Cluster	dse-k8s-eqiad
Kubernetes Namespace	mpic-next
Chart	https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/charts/mpic/
Helmfiles	https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/helmfile.d/dse-k8s-services/mpic-next/
Docker image	https://docker-registry.wikimedia.org/repos/data-engineering/mpic/tags/
Internal service DNS	mpic-next.svc.eqiad.wmnet
Public service URL	https://mpic-next.wikimedia.org
Logs	https://logstash.wikimedia.org/app/dashboards#/view/f9a14f00-dd82-11ef-ae49-21d4ac0f91a0
Metrics	https://grafana.wikimedia.org/d/ee2057f3-eb34-45a7-a48b-489e3ff0b2ec/mpic?orgId=1
Monitors	TBD
Application documentation	https://wikitech.wikimedia.org/wiki/Metrics_Platform/MPIC
Paging	false
Deployment Phabricator ticket	https://phabricator.wikimedia.org/T361335

mpic (production)

Attribute	Value
Owner	Data Products
Kubernetes Cluster	dse-k8s-eqiad
Kubernetes Namespace	mpic
Chart	https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/charts/mpic/
Helmfiles	https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/helmfile.d/dse-k8s-services/mpic
Docker image	https://docker-registry.wikimedia.org/repos/data-engineering/mpic/tags
Internal service DNS	mpic.svc.eqiad.wmnet
Public service URL	https://mpic.wikimedia.org
Logs	https://logstash.wikimedia.org/app/dashboards#/view/d1587980-dd85-11ef-ae49-21d4ac0f91a0
Metrics	https://grafana.wikimedia.org/d/ee2057f3-eb34-45a7-a48b-489e3ff0b2ec/mpic?orgId=1&var-service=mpic
Monitors	TBD
Application documentation	https://wikitech.wikimedia.org/wiki/Metrics_Platform/MPIC
Paging	false
Deployment Phabricator ticket	https://phabricator.wikimedia.org/T361335

Deployment

Note that:

• mpic-next is the staging environment
• mpic is the production environment

# Do the following once you have prepared the change and the change has been reviewed and merged
ssh deployment.[eqiad|codfw].wmnet
cd /srv/deployment-charts
git log -n 1
# Check that the change has been pulled
cd helmfile/dse-k8s-services/<environment>
# Check changes and deploy
helmfile -e dse-k8s-eqiad diff
helmfile -e dse-k8s-eqiad -i apply

Troubleshooting

Note that:

• mpic-next is the staging environment
• mpic is the production environment
• mpic-staging is the container name for staging environment (mpic-next)
• mpic-production is the container name for production environment (mpic)

Check pods

$ ssh deployment.[eqiad|codfw].wmnet
$ kube_env <environment> dse-k8s-eqiad
$ kubectl get pods -w
NAME                            READY   STATUS   RESTARTS   AGE
mpic-staging-57d4bbb6bf-z2p5k   1/2     Error    0          10s
mpic-staging-57d4bbb6bf-z2p5k   1/2     Error    1 (1s ago)   10s
mpic-staging-57d4bbb6bf-z2p5k   1/2     CrashLoopBackOff   1 (1s ago)   11s

See logs

$ ssh deployment.[eqiad|codfw].wmnet
$ kube_env <environment> dse-k8s-eqiad
$ kubectl get pods -w
$ kubectl logs -f <pod_name> -c <container_name> --tail=10
{"level":50,"time":1714483575453,"pid":1,"hostname":"mpic-staging-66554859c4-p9m28","msg":"Error while setting an env variable. There is no value for SAL_PASSWORD"}
{"level":30,"time":1714483575592,"pid":1,"hostname":"mpic-staging-66554859c4-p9m28","msg":"Connected to the database at an-mariadb1001.eqiad.wmnet:mpic_staging"}
{"level":30,"time":1714483575600,"pid":1,"hostname":"mpic-staging-66554859c4-p9m28","msg":"mpic has started listening on port 8080"}