Portal:Toolforge/Admin/Runbooks/k8s-haproxy
These tests are defined here: https://gerrit.wikimedia.org/g/operations/puppet/+/c85d36363c69c1d01092726af556812138475857/modules/profile/manifests/toolforge/k8s/haproxy.pp#56
These checks verify the behavior of the toolforge web proxy. There are two different endpoints tested:
This tests a known, minimal k8s webservice that when working properly returns 'ok' at the healthz route.
When this alert fires it is probably because the tool has crashed. Restart on a toolforge bastion with:
$ sudo su - # become admin # webservice stop # and now wait a minute, just doing a quick restart seems insufficient # webservice start # tail -f error.log # should be fairly quiet
As explained by the endpoint, this is a check of the proxy behavior when accessing a non-existent tool. It ought fall through to the 404 handler and report a useful "this tool does not exist" message.
As of 2022-09-02 this test is known to be flaky due to the 404 handler being a bit broken. A (apparently insufficient) attempt to fix this is https://gerrit.wikimedia.org/r/c/operations/puppet/+/826779/