Data Gateway

This documentation is a stub. Please expand it.

The Data Gateway Service is an abstraction that sits between consumers of published datasets, and the underlying database (currently Cassandra, though other databases are possible as well). It decouples consumers from the database(s), providing them a contract consisting of an HTTP interface and JSON-encoded results, (as opposed to an implementation-specific driver and/or idiomatic client library).

Datasets

Service endpoints

https://data-gateway.discovery.wmnet:30443 (production)

https://data-gateway.k8s-staging.discovery.wmnet:30443 (staging)

Deployment

FIXME: Do.

See: Kubernetes/Deployments

Monitoring & debugging

/healthz

The /healthz endpoint is used for k8s readiness testing, but returns a JSON-encoded object with useful meta-data.

deploy2002:~$ curl https://data-gateway.discovery.wmnet:30443/healthz; echo
{
  "version": "v1.0.10",
  "build_date": "2024-07-23T00:13:01:UTC",
  "build_host": "buildkitsandbox",
  "go_version": "go1.21.8"
}
deploy2002:~$

/metrics

The /metrics endpoint returns Prometheus metrics.

eevans@deploy2002:~$ curl https://data-gateway.discovery.wmnet:30443/metrics 2>/dev/null
[ ... ]
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes 1.8446744073709552e+19
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 7350
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0
eevans@deploy2002:~$

Logging