Nova Resource:Toolsbeta/SAL

2024-11-23

• 07:20 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder (T358225)
• 07:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder (T358225)

2024-11-20

• 15:06 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 15:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 14:56 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 14:54 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 14:53 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 14:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 14:31 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 14:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 14:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
• 14:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 14:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
• 14:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 14:03 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission
• 14:03 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 12:02 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
• 11:58 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 00:15 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission (T362867)
• 00:09 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission (T362867)

2024-11-19

• 21:44 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 21:38 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 21:25 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
• 21:19 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
• 20:43 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
• 20:38 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
• 20:36 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-emailer
• 20:30 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
• 20:20 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api (T362867)
• 20:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api (T362867)
• 20:09 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico (T362867)
• 20:04 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T362867)
• 20:00 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics (T362867)
• 19:54 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics (T362867)
• 19:47 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission (T362867)
• 19:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission (T362867)
• 19:38 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission (T362867)
• 19:32 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission (T362867)
• 19:30 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission (T362867)
• 19:24 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission (T362867)
• 19:23 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission (T362867)
• 19:17 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission (T362867)
• 19:17 raymond-ndibe@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component ingress-admission (T362867)
• 19:16 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission (T362867)
• 15:37 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 15:31 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 10:25 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:13 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:11 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:11 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:10 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-webservice
• 10:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-webservice

2024-11-18

• 14:32 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
• 14:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:57 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
• 10:53 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer

2024-11-14

• 16:33 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-cli
• 16:29 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-cli
• 16:28 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 16:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 12:56 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
• 12:50 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice

2024-11-12

• 13:58 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 13:54 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 13:54 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 13:51 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 13:47 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 13:42 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 13:41 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 13:38 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 13:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 13:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 09:39 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
• 09:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 09:32 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 09:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 09:31 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 09:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice

2024-11-11

• 17:34 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 17:34 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 16:23 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 16:17 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 16:06 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:06 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:04 dcaro@cloudcumin1001: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
• 16:04 dcaro@cloudcumin1001: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:03 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:02 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:02 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:02 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:01 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:01 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:01 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:00 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 16:00 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 16:00 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 15:59 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 15:27 dcaro@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component components-api
• 15:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 15:06 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 15:05 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 15:03 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99)
• 15:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 14:33 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 14:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 14:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 14:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics
• 14:09 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component wmcs-k8s-metrics
• 14:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics
• 13:53 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 13:48 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 13:43 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 13:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 13:41 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 13:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-11-07

• 15:56 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 15:51 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 14:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 14:30 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 14:30 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 14:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 13:50 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 13:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api

2024-11-06

• 16:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 16:16 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 16:15 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 16:12 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 15:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 15:31 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 07:51 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
• 07:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 07:31 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 07:31 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 07:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 07:07 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-11-05

• 17:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 17:06 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 09:34 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 09:28 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 08:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 08:27 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics
• 08:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
• 08:05 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
• 07:44 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico
• 07:39 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico

2024-11-04

• 16:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 16:15 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 12:44 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 12:38 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 11:51 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
• 11:45 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
• 11:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 11:05 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
• 10:43 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
• 10:37 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
• 10:24 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
• 10:20 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 09:47 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
• 09:42 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
• 09:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 09:16 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-10-30

• 15:00 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-9 from 1.27.16 to 1.28.14 (T362867)
• 14:59 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-9 from 1.27.16 to 1.28.14 (T362867)
• 14:59 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-11 from 1.27.16 to 1.28.14 (T362867)
• 14:58 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-11 from 1.27.16 to 1.28.14 (T362867)
• 14:58 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-10 from 1.27.16 to 1.28.14 (T362867)
• 14:57 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-10 from 1.27.16 to 1.28.14 (T362867)
• 14:54 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-13 from 1.27.16 to 1.28.14 (T362867)
• 14:53 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-13 from 1.27.16 to 1.28.14 (T362867)
• 14:53 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-12 from 1.27.16 to 1.28.14 (T362867)
• 14:52 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-12 from 1.27.16 to 1.28.14 (T362867)
• 14:52 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-9 from 1.27.16 to 1.28.14 (T362867)
• 14:51 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-9 from 1.27.16 to 1.28.14 (T362867)
• 14:51 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-8 from 1.27.16 to 1.28.14 (T362867)
• 14:50 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-8 from 1.27.16 to 1.28.14 (T362867)
• 14:50 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-7 from 1.27.16 to 1.28.14 (T362867)
• 14:49 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-7 from 1.27.16 to 1.28.14 (T362867)
• 14:49 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-5 from 1.27.16 to 1.28.14 (T362867)
• 14:48 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-5 from 1.27.16 to 1.28.14 (T362867)
• 14:33 root@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-12 from 1.27.16 to 1.28.14 (T362867)
• 14:28 root@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-12 from 1.27.16 to 1.28.14 (T362867)
• 14:20 root@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-11 from 1.27.16 to 1.28.14 (T362867)
• 14:14 root@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-11 from 1.27.16 to 1.28.14 (T362867)
• 13:16 root@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade from 1.27.16 to 1.28.14 (T362867)
• 13:16 root@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.27.16 to 1.28.14 (T362867)

2024-10-29

• 09:09 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.vps.create_project (exit_code=99) for project toolsbeta in eqiad1
• 09:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.vps.create_project for project toolsbeta in eqiad1

2024-10-16

• 09:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 09:17 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 08:57 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 08:52 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api

2024-10-15

• 17:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 17:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 16:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 16:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-10-10

• 08:20 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
• 08:19 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld

2024-10-09

• 09:24 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 09:16 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-10-08

• 17:43 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:42 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:41 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
• 17:35 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:35 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:34 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:17 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:17 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:10 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:09 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 17:09 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 17:04 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 16:59 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 16:58 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 12:59 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld (T376710)
• 12:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld (T376710)
• 12:52 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld (T376710)
• 12:52 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld (T376710)
• 08:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:08 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 08:08 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:08 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-kubeusers
• 08:07 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 08:03 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 08:03 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain_kubeusers
• 08:03 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain_kubeusers

2024-10-04

• 11:50 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 11:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 11:37 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 11:31 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api

2024-10-03

• 14:04 dcaro: deploying tekton upgrade (builds-builder + builds-api https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/531) T374908
• 14:03 dcaro: deploying tekton upgrade (builds-builder + builds-api https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/531)

2024-10-01

• 10:45 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 10:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 10:27 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 10:18 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 10:13 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 10:08 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 10:06 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 10:00 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 09:40 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 09:34 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission

2024-09-28

• 00:06 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli
• 00:01 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli

2024-09-27

• 23:51 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
• 23:44 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld

2024-09-26

• 16:39 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 16:33 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
• 15:57 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
• 15:51 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
• 15:29 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission (T359641)
• 15:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission (T359641)
• 10:20 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli
• 10:12 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli
• 10:04 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-cli
• 09:59 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli
• 07:59 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-cli
• 07:56 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component builds-cli
• 07:45 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
• 07:40 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 06:52 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 06:52 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
• 06:44 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component toolforge-weld
• 06:43 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld

2024-09-25

• 14:15 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-nfs-10
• 08:26 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:24 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 07:46 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-nfs-7
• 07:32 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 07:15 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-nfs-7
• 07:02 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 06:55 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 06:48 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 06:33 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 06:32 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 06:25 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 06:23 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 06:16 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 06:06 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 05:59 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 05:50 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 05:49 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 05:48 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 05:48 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-10.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 05:38 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 05:38 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-10
• 05:38 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-10
• 05:37 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-10
• 05:37 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-10
• 05:33 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the toolsbeta cluster
• 05:32 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 05:32 raymond-ndibe@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=97) for a worker-nfs role in the toolsbeta cluster
• 05:29 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 05:18 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 05:17 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 05:16 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 05:15 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 04:42 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 04:40 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-09-24

• 22:03 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers (T375157)
• 21:56 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers (T375157)
• 21:41 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component kyverno (T359641)
• 21:35 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component kyverno (T359641)

2024-09-21

• 03:23 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 03:18 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7
• 03:17 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-worker-nfs-7
• 03:12 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-nfs-7

2024-09-20

• 19:30 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico (T341066)
• 19:26 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T341066)
• 19:24 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component calico (T341066)
• 19:19 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T341066)
• 00:30 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics (T359641)
• 00:25 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics (T359641)

2024-09-19

• 17:41 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli (T341066)
• 17:35 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli (T341066)
• 17:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli (T341066)
• 17:28 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli (T341066)
• 17:27 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli (T341066)
• 17:27 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli (T341066)
• 17:26 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-cli (T341066)
• 17:26 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli (T341066)
• 14:47 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api (T341066)
• 14:42 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
• 14:10 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api (T341066)
• 14:05 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)

2024-09-11

• 12:56 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0)
• 12:55 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node
• 12:53 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0)
• 12:52 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node
• 12:52 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0)
• 12:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node
• 12:26 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the toolsbeta cluster
• 12:26 wmbot~dcaro@urcuchillay: Added a new k8s ingress toolsbeta-test-k8s-ingress-11.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 12:17 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 11:44 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the toolsbeta cluster
• 11:44 wmbot~dcaro@urcuchillay: Added a new k8s ingress toolsbeta-test-k8s-ingress-10.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 11:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 10:34 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the toolsbeta cluster
• 10:34 wmbot~dcaro@urcuchillay: Added a new k8s ingress toolsbeta-test-k8s-ingress-9.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 10:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 10:15 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 09:47 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0)
• 09:45 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node
• 09:43 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0)
• 09:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node
• 09:24 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the toolsbeta cluster
• 09:24 wmbot~dcaro@urcuchillay: Added a new k8s worker toolsbeta-test-k8s-worker-13.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 09:14 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the toolsbeta cluster
• 09:09 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the toolsbeta cluster
• 09:09 wmbot~dcaro@urcuchillay: Added a new k8s worker toolsbeta-test-k8s-worker-12.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 08:59 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the toolsbeta cluster

2024-09-10

• 14:46 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 14:46 dcaro@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-7.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 14:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster (T359641)
• 14:35 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 14:35 dcaro@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-6.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 14:25 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster (T359641)
• 14:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 14:21 dcaro@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-5.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 14:11 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster (T359641)

2024-09-09

• 16:14 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component cert-manager
• 16:09 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 14:29 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-11 from 1.26.15 to 1.27.16 (T359641)
• 14:29 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-11 from 1.26.15 to 1.27.16 (T359641)

2024-09-06

• 09:17 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component cert-manager
• 09:14 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 09:13 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component cert-manager
• 09:10 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 09:00 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component cert-manager
• 08:55 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 08:34 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 08:29 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 08:28 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
• 08:26 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 06:24 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 06:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice

2024-09-05

• 20:51 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component cert-manager
• 20:50 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 20:37 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component cert-manager
• 20:37 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 20:36 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component cert-manager
• 20:36 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager
• 17:41 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-control-9
• 17:39 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-9
• 17:39 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the toolsbeta cluster
• 17:39 wmbot~dcaro@urcuchillay: Added a new k8s control toolsbeta-test-k8s-control-12.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 17:28 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 17:23 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-control-8
• 17:22 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-8
• 17:18 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-control-8
• 17:18 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-8
• 17:18 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-control-7
• 17:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-7
• 14:53 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-8 from 1.26.15 to 1.27.16 (T359641)
• 14:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.26.15 to 1.27.16 (T359641)
• 14:45 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-7 from 1.26.15 to 1.27.16 (T359641)
• 14:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.26.15 to 1.27.16 (T359641)
• 14:25 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade from 1.26.15 to 1.27.16 (T359641)
• 14:23 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.26.15 to 1.27.16 (T359641)

2024-09-04

• 14:01 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 13:57 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 13:57 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 13:56 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 13:55 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 13:49 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
• 13:34 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
• 13:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
• 13:18 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
• 13:18 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
• 12:51 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 12:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
• 12:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 12:44 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
• 11:20 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
• 11:12 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-09-03

• 20:01 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 19:56 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
• 19:47 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
• 19:40 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
• 19:28 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
• 19:23 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
• 19:17 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
• 19:12 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 19:07 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission
• 19:01 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 18:50 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-admission
• 18:44 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 16:53 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the toolsbeta cluster
• 16:53 wmbot~dcaro@urcuchillay: Added a new k8s control toolsbeta-test-k8s-control-11.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 16:42 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 16:40 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the toolsbeta cluster
• 16:40 wmbot~dcaro@urcuchillay: Added a new k8s control toolsbeta-test-k8s-control-10.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 16:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 16:26 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a control role in the toolsbeta cluster
• 16:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 15:27 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a control role in the toolsbeta cluster
• 15:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 15:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component kyverno
• 15:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno
• 14:58 dcaro@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component kyverno
• 14:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno
• 14:54 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component kyverno
• 14:50 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 14:44 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 14:44 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
• 14:44 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 14:44 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
• 14:44 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 14:32 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno
• 14:32 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component kyverno
• 14:32 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno
• 14:11 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
• 13:33 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
• 13:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
• 13:10 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 13:05 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 12:58 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 12:53 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 12:50 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
• 12:48 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 10:27 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
• 10:22 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
• 10:10 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 10:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 09:45 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
• 09:39 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder

2024-09-02

• 09:57 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 09:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 09:34 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
• 09:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
• 09:14 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 09:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 08:55 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 08:48 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api

2024-08-29

• 16:24 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
• 16:19 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder

2024-08-28

• 17:22 andrewbogott: shutting down toolsbeta-harbor-2 to (I hope) quiet alerts. Raymond can start this up again when he's back.
• 14:04 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-11 from 1.25.16 to 1.26.15
• 14:03 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-11 from 1.25.16 to 1.26.15
• 14:02 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-10 from 1.25.16 to 1.26.15
• 14:01 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-10 from 1.25.16 to 1.26.15
• 14:00 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-3 from 1.25.16 to 1.26.15
• 13:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-3 from 1.25.16 to 1.26.15
• 13:56 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-2 from 1.25.16 to 1.26.15
• 13:55 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-2 from 1.25.16 to 1.26.15
• 13:53 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-1 from 1.25.16 to 1.26.15
• 13:52 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-1 from 1.25.16 to 1.26.15
• 13:51 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-9 from 1.25.16 to 1.26.15
• 13:32 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-9 from 1.25.16 to 1.26.15
• 13:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-8 from 1.25.16 to 1.26.15
• 13:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.25.16 to 1.26.15
• 13:18 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-8 from 1.25.16 to 1.26.15
• 13:18 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.25.16 to 1.26.15
• 13:17 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-7 from 1.25.16 to 1.26.15
• 12:49 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.25.16 to 1.26.15
• 12:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade from 1.25.16 to 1.26.15
• 12:45 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.25.16 to 1.26.15
• 12:43 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=99) for cluster toolsbeta upgrade from 1.25.16 to 1.26.15
• 12:43 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.25.16 to 1.26.15
• 06:30 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-nginx
• 06:30 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-nginx
• 06:25 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-nginx
• 06:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-nginx

2024-08-27

• 08:46 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico
• 08:41 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico
• 08:28 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component calico
• 08:27 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component calico

2024-08-26

• 09:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 09:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-08-21

• 05:37 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 05:31 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 05:19 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
• 05:14 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
• 05:13 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-builder
• 05:04 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
• 04:52 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 04:45 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 04:18 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
• 04:12 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
• 04:03 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
• 03:56 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
• 03:41 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 03:35 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 03:12 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 03:06 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 02:59 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
• 02:55 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 02:53 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
• 02:48 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 01:54 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
• 01:49 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 01:46 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 01:42 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.run_tests
• 01:39 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 01:38 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api

2024-08-13

• 09:48 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 09:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 09:42 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
• 09:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 09:40 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
• 09:38 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api

2024-08-12

• 15:19 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 15:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 12:16 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 12:11 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 12:05 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
• 12:03 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 12:00 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 12:00 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 11:42 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
• 11:37 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 11:36 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 11:35 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:37 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
• 10:31 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
• 10:24 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
• 10:18 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 10:01 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
• 09:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
• 09:50 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 09:45 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 09:41 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 09:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 09:14 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 09:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 09:09 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
• 09:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
• 09:02 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
• 09:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-08-08

• 16:51 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
• 16:45 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
• 16:42 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api
• 16:40 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
• 16:28 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
• 16:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 16:13 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
• 16:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
• 16:04 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 15:58 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 15:29 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 15:29 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 15:28 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 15:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 15:28 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components
• 15:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components
• 15:27 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component compontents
• 15:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component compontents

2024-08-06

• 13:04 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 13:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 13:01 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 13:00 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 12:55 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 12:54 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api

2024-08-05

• 18:28 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 18:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 18:26 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 18:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 18:22 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 18:21 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 18:10 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 18:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 18:05 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 18:04 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 17:58 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 17:57 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 17:57 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 17:57 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 17:56 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 17:50 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 17:49 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
• 17:48 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 17:48 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
• 17:47 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
• 16:52 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 16:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 16:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 16:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:56 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 15:55 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:55 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 15:55 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:54 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 15:53 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:53 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 15:52 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:52 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:52 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:52 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:51 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:41 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 15:36 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:29 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:23 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:14 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:14 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:13 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:12 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:12 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:11 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:11 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:11 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:09 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:04 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:04 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:03 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 15:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:03 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.toolforge.run_tests (exit_code=1)
• 15:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 15:01 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 15:00 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:59 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.run_tests (exit_code=0)
• 14:58 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:58 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:57 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:54 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:54 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:50 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:31 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:30 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:30 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:29 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:29 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:28 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:27 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:27 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:17 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 14:17 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.run_tests (exit_code=99)
• 14:17 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.run_tests
• 13:34 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component components-api
• 13:34 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component components-api
• 11:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 11:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 09:07 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 09:07 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 08:31 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 08:30 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-08-01

• 15:26 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component components-api
• 15:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component components-api

2024-07-31

• 20:32 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
• 20:30 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
• 12:52 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component calico
• 12:52 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component calico
• 12:29 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component calico
• 12:28 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component calico
• 11:22 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 11:22 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 11:16 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component components-api
• 11:16 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component components-api

2024-07-30

• 17:34 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 17:07 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
• 17:05 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli

2024-07-29

• 18:22 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 18:21 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 18:02 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 18:01 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 16:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 16:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 16:32 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 16:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 16:10 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 16:09 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 16:07 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 16:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 16:03 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 16:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 15:25 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 15:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 15:16 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 15:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 15:15 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 15:15 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 15:13 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 15:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 15:03 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 15:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 14:42 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 14:42 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 12:43 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
• 12:42 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
• 12:41 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component jobs-cli
• 12:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
• 12:39 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component envvars-cli
• 12:39 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-cli
• 12:38 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component envvars-cli
• 12:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-cli
• 11:42 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:27 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:25 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 11:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:11 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:10 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:08 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:05 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:04 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:03 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 11:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 11:00 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 10:59 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 10:25 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 10:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 10:21 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 10:21 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 10:19 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 10:18 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 10:02 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 10:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 10:02 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 10:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 09:59 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 09:59 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 09:57 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 09:56 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 09:56 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
• 09:55 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 09:54 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 09:54 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
• 09:53 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
• 09:53 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli

2024-07-25

• 15:04 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 15:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 08:05 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 08:04 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics

2024-07-24

• 08:58 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 08:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
• 06:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
• 06:45 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission

2024-07-23

• 14:01 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 14:01 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 12:53 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 12:53 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 12:16 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 12:16 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 12:10 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 12:10 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 11:58 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 11:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 08:00 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 08:00 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-07-22

• 15:39 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 15:39 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 09:32 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 09:32 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-07-18

• 14:35 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 14:35 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 08:44 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 08:44 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-07-17

• 14:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 14:45 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:33 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:32 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:23 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 10:23 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 10:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 10:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 08:54 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 08:54 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 08:54 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component jobs-api
• 08:54 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 08:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 08:22 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
• 08:20 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 08:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
• 08:14 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 08:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx

2024-07-16

• 16:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 16:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 14:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 14:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 11:34 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 11:34 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:14 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
• 10:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission
• 08:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
• 08:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission

2024-07-15

• 14:33 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 14:33 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 11:35 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 11:35 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 08:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 07:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2024-07-12

• 10:13 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-8 from 1.24.17 to 1.25.16
• 10:12 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-8 from 1.24.17 to 1.25.16
• 10:09 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-7 from 1.24.17 to 1.25.16
• 10:08 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-7 from 1.24.17 to 1.25.16
• 10:07 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-worker-ingress-7 from 1.24.17 to 1.25.16
• 10:07 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-ingress-7 from 1.24.17 to 1.25.16
• 10:00 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-4 from 1.24.17 to 1.25.16
• 09:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-4 from 1.24.17 to 1.25.16
• 09:53 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-3 from 1.24.17 to 1.25.16
• 09:51 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-3 from 1.24.17 to 1.25.16
• 09:48 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-2 from 1.24.17 to 1.25.16
• 09:47 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-2 from 1.24.17 to 1.25.16
• 09:43 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-11 from 1.24.17 to 1.25.16
• 09:42 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-11 from 1.24.17 to 1.25.16

2024-07-11

• 17:46 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 17:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 12:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-6 from 1.24.17 to 1.25.16
• 12:54 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-6 from 1.24.17 to 1.25.16
• 12:53 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-10 from 1.24.17 to 1.25.16
• 12:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-10 from 1.24.17 to 1.25.16
• 12:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-1 from 1.24.17 to 1.25.16
• 12:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-1 from 1.24.17 to 1.25.16
• 12:48 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-9 from 1.24.17 to 1.25.16
• 12:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-9 from 1.24.17 to 1.25.16
• 12:39 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-8 from 1.24.17 to 1.25.16
• 12:31 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.24.17 to 1.25.16
• 12:31 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-7 from 1.24.17 to 1.25.16
• 12:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.24.17 to 1.25.16
• 12:12 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-worker-4 from 1.24.17 to 1.25.16
• 12:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-worker-4 from 1.24.17 to 1.25.16
• 12:10 arturo: upgrading k8s cluster to 1.25 (control plane) T369168
• 12:06 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade from 1.24.17 to 1.25.16
• 12:01 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.24.17 to 1.25.16
• 11:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
• 11:54 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission

2024-07-10

• 17:05 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 17:05 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 16:47 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 16:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 15:58 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 15:58 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 15:15 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:15 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:48 arturo: manually deleted tool-test8 and tool-test8xx k8s namespaces to have them recreated by maintain-kubeusers
• 12:44 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:44 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:01 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 10:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-07-09

• 14:20 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 14:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 14:17 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 14:17 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-07-08

• 13:59 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 13:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 13:28 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 13:28 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 13:06 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 13:05 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 12:26 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
• 12:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
• 11:29 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 11:29 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 08:36 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 08:35 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-07-05

• 12:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 12:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 12:33 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 12:33 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 01:42 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 01:41 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 01:41 wmbot~raymond@ubuntu: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component jobs-api
• 01:41 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-07-04

• 17:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 17:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 17:04 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component api-gateway
• 17:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 12:57 arturo: updating kubelet flags T355881
• 10:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 10:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 09:38 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 09:38 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 09:32 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 09:32 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 09:25 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 09:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 07:45 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 07:45 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-07-03

• 12:19 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 12:19 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 10:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 09:51 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 09:50 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2024-07-02

• 17:00 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 17:00 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 16:48 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 16:48 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 14:47 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 14:47 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 14:46 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component envvars-api
• 14:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 11:54 arturo: cleanup extra redundant cert-signing settings from controller-manager arguments
• 10:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 10:53 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 10:53 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 10:53 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 09:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 09:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 09:22 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component cert-manager
• 09:22 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component cert-manager
• 09:08 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 09:08 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-07-01

• 15:39 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 15:39 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 15:31 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:31 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 14:57 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 14:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 14:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 14:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 13:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
• 13:14 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
• 13:02 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 13:02 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission

2024-06-28

• 11:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 11:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 09:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 09:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 09:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 09:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 09:30 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 09:30 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 09:24 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 09:24 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-06-27

• 16:40 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-test-k8s-etcd-26
• 16:37 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-etcd-26
• 16:36 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-test-k8s-etcd-25
• 16:34 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-etcd-25
• 15:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component cert-manager
• 14:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component cert-manager
• 14:53 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-test-k8s-etcd-23
• 14:49 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-etcd-23
• 14:49 andrew@cloudcumin1001: END (ERROR) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=97) for server toolsbeta-test-k8s-etcd-23
• 14:47 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 14:47 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
• 14:44 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-etcd-23
• 14:43 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server toolsbeta-test-k8s-etcd-23
• 14:43 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 14:43 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
• 14:36 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-etcd-23
• 13:47 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server toolsbeta-test-k8s-etcd-23
• 13:47 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-etcd-23
• 10:59 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:30 arturo: disabled PodSecurityPolicy admission plugin from kubeadm configmap (T368142)
• 09:28 arturo: disabled PodSecurityPolicy admission plugin from apiserver static pod manifests (T368142)
• 08:57 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:57 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 08:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-26

• 10:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:39 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:17 arturo: deploying toolforge-webservice 0.103.9 (T368463)
• 09:15 arturo: setting kyverno policies to Enforce (T368141)
• 09:15 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:15 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-25

• 12:10 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 12:10 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 11:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.migrate_floating_ip (exit_code=0) for address 185.15.56.33 to server 'toolsbeta-proxy-5'
• 11:06 taavi@cloudcumin1001: START - Cookbook wmcs.vps.migrate_floating_ip for address 185.15.56.33 to server 'toolsbeta-proxy-5'
• 11:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.migirate_floating_ip (exit_code=0) for address 185.15.56.33 to server 'toolsbeta-proxy-6'
• 11:02 taavi@cloudcumin1001: START - Cookbook wmcs.vps.migirate_floating_ip for address 185.15.56.33 to server 'toolsbeta-proxy-6'
• 09:42 arturo: deploy toolforge-webservice 0.103.8 (T362050)
• 09:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 09:23 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-06-24

• 15:44 arturo: deploy toolforge-webservice 0.103.7 (T362050)
• 12:09 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 12:09 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 11:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 11:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 11:44 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 11:44 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 10:05 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:05 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-21

• 03:11 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs (exit_code=0) for server tbd
• 02:59 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs for server tbd

2024-06-20

• 14:23 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_project_to_ovs (exit_code=1)
• 14:03 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_project_to_ovs
• 12:53 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 12:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 09:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:07 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:06 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-19

• 09:55 arturo: merging k8s HAproxy change https://gerrit.wikimedia.org/r/c/operations/puppet/+/1047113
• 04:17 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 04:16 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 04:15 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 04:14 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-06-17

• 13:28 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-test-k8s-ingress-7
• 13:26 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-ingress-7
• 12:05 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-test-k8s-worker-10
• 12:04 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-worker-10
• 11:53 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-test-k8s-haproxy-5
• 11:52 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-test-k8s-haproxy-5
• 11:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-legacy-redirector-2
• 11:41 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-legacy-redirector-2
• 11:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-harbor-1
• 11:40 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-harbor-1
• 11:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-puppetserver-1
• 11:33 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-puppetserver-1
• 11:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-puppetdb-03
• 11:31 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-puppetdb-03
• 11:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-proxy-6
• 11:29 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-proxy-6
• 11:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-proxy-5
• 11:28 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-proxy-5
• 11:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-prometheus-1
• 11:26 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-prometheus-1
• 11:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-mail-2
• 11:25 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-mail-2
• 11:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-bastion-6
• 11:22 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-bastion-6
• 10:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-docker-imagebuilder-2
• 10:51 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-docker-imagebuilder-2
• 10:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-acme-chief-2
• 10:49 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-acme-chief-2
• 10:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-static-2
• 10:48 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-static-2

2024-06-14

• 13:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-sgebastion-05
• 13:10 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-sgebastion-05
• 13:09 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server toolsbeta-redis-1
• 13:08 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server toolsbeta-redis-1
• 08:02 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 08:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 07:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 07:21 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-06-12

• 17:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 17:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 17:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 17:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 16:27 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 16:26 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 15:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 15:22 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 15:21 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 15:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:02 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:31 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 13:31 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 11:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 11:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-11

• 11:35 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 11:35 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:40 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:26 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-07

• 11:28 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
• 11:28 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
• 11:27 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 11:27 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:02 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 10:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 09:50 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 09:50 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-06-06

• 14:16 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 14:16 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 14:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 14:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:01 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-05

• 16:04 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 16:04 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 15:27 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:26 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:44 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 12:44 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-06-04

• 16:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 16:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:18 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 12:18 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 12:18 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:18 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:17 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 12:17 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 12:06 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:06 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:32 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:32 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:25 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:25 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 08:05 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 08:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-06-03

• 16:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 16:21 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 14:11 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 14:10 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:36 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:35 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:18 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:17 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 12:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:02 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:25 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 09:25 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 08:56 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 08:56 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 08:34 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 08:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 08:16 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 08:16 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission

2024-05-30

• 12:05 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:05 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 09:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-05-29

• 14:56 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 14:56 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 07:45 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 07:45 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 03:00 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 03:00 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-05-28

• 12:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:43 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-05-27

• 16:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 16:02 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 15:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:54 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 15:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-05-25

• 21:10 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 21:09 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 20:37 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 20:36 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-05-23

• 13:13 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 13:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-05-15

• 10:25 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:25 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-05-14

• 13:25 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 13:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 13:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 13:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-05-10

• 13:57 taavi: renew k8s prometheus certificate

2024-05-07

• 16:18 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 16:18 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 15:29 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99)
• 15:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 12:07 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 12:06 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-05-06

• 11:29 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 11:29 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 09:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 09:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 08:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 08:12 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 07:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 07:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-05-04

• 15:16 taavi: $ sudo docker exec -it striker-toolsbeta.service poetry run python3 manage.py loaddata software_license.json
• 14:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 14:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx

2024-05-03

• 15:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 15:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 12:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 12:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:16 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:16 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-04-30

• 10:52 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 10:52 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-04-26

• 08:59 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 08:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 08:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-04-25

• 12:55 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 12:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-24

• 15:25 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 15:25 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-04-18

• 09:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 09:23 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 08:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 08:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-15

• 20:26 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 20:26 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 18:21 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 18:20 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 17:51 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 17:50 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 17:31 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 17:30 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 13:41 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 13:41 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 13:38 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
• 13:38 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 13:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 13:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 11:42 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 11:42 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 11:02 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 11:02 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:58 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 10:58 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 08:24 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 08:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-04-12

• 15:45 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 15:32 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 15:21 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 15:15 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 15:14 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 15:08 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:59 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 14:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 14:45 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 14:40 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:39 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 14:34 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:17 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 14:09 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:08 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 14:00 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 10:11 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
• 10:10 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission
• 09:31 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 09:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 09:30 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component volume-admisison
• 09:30 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admisison
• 09:24 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 09:23 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 05:01 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 04:51 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 04:48 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 04:48 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 04:47 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 04:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 04:46 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 04:37 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 04:35 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 04:22 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 03:37 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 03:36 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 03:17 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 03:06 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 02:58 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 02:58 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 02:21 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 02:05 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 01:28 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 01:19 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 01:18 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component calico
• 01:18 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 01:17 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component calico
• 01:17 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
• 01:16 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
• 01:16 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
• 01:15 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
• 01:15 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
• 01:14 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
• 01:11 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 01:10 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 01:09 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 01:09 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 00:58 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 00:56 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 00:43 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node

2024-04-11

• 23:06 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 22:54 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 22:18 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 22:11 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 22:10 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 22:03 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 22:01 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 22:00 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:49 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 20:39 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:38 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 20:27 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:27 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 20:18 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:17 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 20:17 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:05 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 20:04 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:03 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 20:02 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:02 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 20:01 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:00 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 19:59 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 19:58 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 19:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 19:34 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 19:17 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 19:16 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 19:03 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 18:58 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 18:49 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 17:33 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 17:23 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 17:23 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 17:13 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 17:13 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 17:12 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 17:06 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 16:52 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 16:50 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 16:34 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 16:31 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 16:23 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 16:22 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 16:08 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 08:38 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 08:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 08:37 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 08:37 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-04-10

• 19:01 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 18:54 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 18:48 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 18:45 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 18:43 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 18:36 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 18:16 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 18:13 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node
• 02:56 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 02:48 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 02:26 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 02:25 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 00:41 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 00:32 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 00:16 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 00:07 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node

2024-04-09

• 23:17 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 23:08 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 23:07 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 23:07 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 22:52 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 22:40 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 22:29 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 22:17 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 22:16 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 22:01 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 21:24 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 21:09 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 21:08 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 21:04 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 21:01 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 20:52 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:44 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 20:44 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 20:30 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 20:21 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 19:52 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 19:43 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 19:42 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 19:28 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 18:55 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 18:39 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 18:17 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 18:09 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 14:08 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 14:08 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 13:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 13:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 11:57 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 11:56 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-08

• 16:08 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 16:01 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 15:51 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 15:44 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 10:17 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-05

• 12:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 12:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-03

• 16:05 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 16:04 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 14:30 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 14:29 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-04-02

• 19:30 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 19:22 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 19:22 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 19:13 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 19:13 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 19:04 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 18:58 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 18:49 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 18:46 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 18:37 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 18:36 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 18:20 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 18:08 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 17:53 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 17:39 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 17:25 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 17:19 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 17:14 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 17:08 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 17:01 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 17:00 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 16:54 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 16:33 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 16:22 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 16:15 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 16:08 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 15:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-localdisk
• 15:31 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-localdisk
• 15:28 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 15:22 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 15:16 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 15:06 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 15:06 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 14:59 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:55 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 14:50 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:44 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 14:33 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 14:26 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 14:20 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 07:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-docker-registry-02
• 07:54 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-docker-registry-02

2024-04-01

• 15:49 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 15:48 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 15:25 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 15:18 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 15:11 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 15:00 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 14:49 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 14:42 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 14:40 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 14:30 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 14:19 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 14:13 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node

2024-03-28

• 17:51 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 17:42 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 17:04 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
• 16:58 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 16:56 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
• 16:50 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
• 16:33 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
• 16:24 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 16:13 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 16:09 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 15:54 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 15:53 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 15:49 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 15:49 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 15:36 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 15:35 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 15:30 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
• 15:29 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
• 15:19 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.etcd.remove_node_from_hiera (exit_code=0) (T349207)
• 15:19 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.remove_node_from_hiera (T349207)
• 14:48 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (exit_code=99)
• 14:48 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (T349207)
• 14:46 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_hiera (exit_code=0) (T349207)
• 14:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_hiera (T349207)
• 14:44 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (exit_code=99)
• 14:43 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (T349207)
• 14:42 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_hiera (exit_code=0) (T349207)
• 14:42 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_hiera (T349207)
• 14:33 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (exit_code=99)
• 14:32 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (T360699)
• 14:30 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (exit_code=99)
• 14:27 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (T360699)
• 14:25 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (exit_code=99)
• 14:25 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.etcd.add_node_to_cluster (T360699)
• 14:01 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-proxy-3
• 14:01 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-proxy-3
• 13:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-proxy-4
• 13:16 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-proxy-4
• 13:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'toolsbeta-proxy'
• 13:11 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'toolsbeta-proxy'
• 13:07 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-proxy-5.toolsbeta.eqiad1.wikimedia.cloud
• 13:05 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-proxy-5.toolsbeta.eqiad1.wikimedia.cloud
• 13:02 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'toolsbeta-proxy'
• 12:58 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'toolsbeta-proxy'

2024-03-27

• 12:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-nfs-2
• 12:27 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-nfs-2

2024-03-26

• 14:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.nfs.migrate_service (exit_code=0)
• 14:28 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service
• 14:22 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99)
• 14:22 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service
• 14:11 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99)
• 14:11 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service
• 14:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.nfs.add_server (exit_code=0)
• 14:03 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.add_server
• 14:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-nfs-3
• 14:02 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-nfs-3
• 14:01 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.add_server (exit_code=99)
• 13:56 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.add_server
• 13:55 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.nfs.add_server (exit_code=97)
• 13:54 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.add_server
• 13:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-nfs-3
• 13:50 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-nfs-3
• 13:34 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99)
• 13:33 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service
• 13:31 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.migrate_service (exit_code=99)
• 13:31 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.migrate_service
• 13:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-nfs-3.toolsbeta.eqiad1.wikimedia.cloud
• 13:22 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-nfs-3.toolsbeta.eqiad1.wikimedia.cloud
• 13:20 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.nfs.add_server (exit_code=99)
• 13:16 taavi@cloudcumin1001: START - Cookbook wmcs.nfs.add_server

2024-03-25

• 18:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-legacy-redirector
• 18:55 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-legacy-redirector

2024-03-22

• 11:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-legacy-redirector-2.toolsbeta.eqiad1.wikimedia.cloud
• 11:19 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-legacy-redirector-2.toolsbeta.eqiad1.wikimedia.cloud

2024-03-21

• 14:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_haproxy_node (exit_code=0) for node toolsbeta-test-k8s-haproxy-4
• 14:10 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_haproxy_node for node toolsbeta-test-k8s-haproxy-4
• 13:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_haproxy_node (exit_code=0) for node toolsbeta-test-k8s-haproxy-3
• 13:38 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_haproxy_node for node toolsbeta-test-k8s-haproxy-3
• 12:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_haproxy_node (exit_code=0)
• 12:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_haproxy_node

2024-03-20

• 15:55 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_haproxy_node (exit_code=0)
• 15:49 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_haproxy_node
• 11:17 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 11:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-03-19

• 10:21 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 10:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-03-18

• 12:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-static-1
• 12:33 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-static-1
• 11:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-acme-chief-2.toolsbeta.eqiad1.wikimedia.cloud
• 11:40 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-acme-chief-2.toolsbeta.eqiad1.wikimedia.cloud
• 10:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-static-2.toolsbeta.eqiad1.wikimedia.cloud
• 10:55 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-static-2.toolsbeta.eqiad1.wikimedia.cloud
• 10:50 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on toolsbeta-static-2.toolsbeta.eqiad1.wikimedia.cloud
• 10:50 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-static-2.toolsbeta.eqiad1.wikimedia.cloud

2024-03-16

• 11:09 taavi: reenable puppet on toolsbeta-test-k8s-control-7/8

2024-03-15

• 10:48 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-03-14

• 15:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-docker-imagebuilder-01
• 15:17 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-docker-imagebuilder-01
• 13:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-8 from 1.23.17 to 1.24.17 (T359638)
• 13:01 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-8 from 1.23.17 to 1.24.17 (T359638)
• 13:01 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-7 from 1.23.17 to 1.24.17 (T359638)
• 13:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-7 from 1.23.17 to 1.24.17 (T359638)
• 13:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-6 from 1.23.17 to 1.24.17 (T359638)
• 12:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-6 from 1.23.17 to 1.24.17 (T359638)
• 12:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-4 from 1.23.17 to 1.24.17 (T359638)
• 12:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-4 from 1.23.17 to 1.24.17 (T359638)
• 12:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-3 from 1.23.17 to 1.24.17 (T359638)
• 12:44 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-3 from 1.23.17 to 1.24.17 (T359638)
• 12:44 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-2 from 1.23.17 to 1.24.17 (T359638)
• 12:43 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-2 from 1.23.17 to 1.24.17 (T359638)
• 12:43 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-1 from 1.23.17 to 1.24.17 (T359638)
• 12:42 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-nfs-1 from 1.23.17 to 1.24.17 (T359638)
• 12:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-11 from 1.23.17 to 1.24.17 (T359638)
• 12:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-11 from 1.23.17 to 1.24.17 (T359638)
• 12:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-10 from 1.23.17 to 1.24.17 (T359638)
• 12:39 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-10 from 1.23.17 to 1.24.17 (T359638)
• 12:36 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-9 from 1.23.17 to 1.24.17 (T359638)
• 12:27 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-9 from 1.23.17 to 1.24.17 (T359638)
• 12:27 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-9 from 1.23.17 to 1.24.17 (T359638)
• 12:27 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-9 from 1.23.17 to 1.24.17 (T359638)
• 11:30 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.restart_static_pods (exit_code=99) for toolsbeta-test-k8s-control-8 (T359638)
• 11:29 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.restart_static_pods for toolsbeta-test-k8s-control-8 (T359638)
• 10:40 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.restart_static_pods (exit_code=99) for toolsbeta-test-k8s-control-8 (T359638)
• 10:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.restart_static_pods for toolsbeta-test-k8s-control-8 (T359638)
• 10:36 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.24.17 (T359638)
• 10:33 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.24.17 (T359638)
• 10:33 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.27.17 (T359638)
• 10:31 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.27.17 (T359638)
• 10:14 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.27.17 (T359638)
• 10:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.27.17 (T359638)
• 10:12 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.27.17 (T359638)
• 10:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-8 from 1.23.17 to 1.27.17 (T359638)
• 09:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 09:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 09:14 aborrero@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=97) for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 09:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)

2024-03-13

• 16:15 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 16:15 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 16:14 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 16:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-control-7 from 1.23.17 to 1.27.17 (T359638)
• 15:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster toolsbeta upgrade from 1.23 to 1.24 (T359638)
• 15:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.23 to 1.24 (T359638)

2024-03-12

• 11:15 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=99) for cluster toolsbeta upgrade from 1.23 to 1.24 (T359638)
• 11:15 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster toolsbeta upgrade from 1.23 to 1.24 (T359638)

2024-03-11

• 16:55 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 16:55 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 16:55 aborrero@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component wmcs-k8s-metrics
• 16:55 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 16:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 16:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics

2024-03-07

• 14:17 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 14:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 13:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 13:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-03-05

• 16:08 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 16:08 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-03-04

• 17:55 bd808@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 17:55 bd808@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-03-01

• 21:14 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 21:13 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-02-28

• 00:39 bd808@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 00:39 bd808@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-02-26

• 13:07 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-docker-imagebuilder-2.toolsbeta.eqiad1.wikimedia.cloud
• 13:06 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-docker-imagebuilder-2.toolsbeta.eqiad1.wikimedia.cloud

2024-02-22

• 13:59 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 13:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 10:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-02-21

• 17:05 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 17:05 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 15:48 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 15:47 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 14:51 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 14:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 14:40 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 14:39 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 14:32 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusers
• 14:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:27 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 13:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:25 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusers
• 13:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusers
• 13:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:20 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusers
• 13:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:20 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusers
• 13:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:16 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component maintain-kubeusers
• 13:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-02-20

• 13:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-6
• 13:48 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=2) for host toolsbeta-test-k8s-control-6
• 13:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-6
• 13:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the toolsbeta cluster
• 13:46 taavi@cloudcumin1001: Added a new k8s control toolsbeta-test-k8s-control-9.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 13:38 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 13:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the toolsbeta cluster
• 13:38 taavi@cloudcumin1001: Added a new k8s worker toolsbeta-test-k8s-worker-11.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 13:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the toolsbeta cluster
• 13:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-9
• 13:26 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=2) for host toolsbeta-test-k8s-worker-9
• 13:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-9
• 13:26 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-test-k8s-worker-9
• 13:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-test-k8s-worker-9
• 13:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the toolsbeta cluster
• 13:26 taavi@cloudcumin1001: Added a new k8s worker toolsbeta-test-k8s-worker-10.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 13:18 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the toolsbeta cluster
• 11:56 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-1
• 11:56 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.k8s.worker.drain for node toolsbeta-test-k8s-worker-nfs-1
• 11:56 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node toolsbeta-test-k8s-worker-nfs-1
• 11:55 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.k8s.worker.drain for node toolsbeta-test-k8s-worker-nfs-1

2024-02-19

• 18:46 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 18:44 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-02-15

• 11:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-control-5
• 11:11 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-5
• 11:09 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-control-5
• 11:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-5
• 11:08 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeta-test-k8s-control-5
• 11:08 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-5
• 11:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the toolsbeta cluster
• 11:06 taavi@cloudcumin1001: Added a new k8s control toolsbeta-test-k8s-control-8.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 10:58 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 10:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-k8s-control-8
• 10:57 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-k8s-control-8
• 10:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-k8s-control-8
• 10:55 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-k8s-control-8
• 10:53 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a control role in the toolsbeta cluster
• 10:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 10:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-k8s-control-8
• 10:45 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-k8s-control-8
• 10:44 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a control role in the toolsbeta cluster
• 10:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster

2024-02-13

• 14:53 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-control-4
• 14:52 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-control-4
• 14:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-ingress-5
• 14:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-ingress-5
• 14:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-ingress-4
• 14:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-ingress-4
• 10:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the toolsbeta cluster
• 10:11 taavi@cloudcumin1001: Added a new k8s ingress toolsbeta-test-k8s-ingress-8.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 10:04 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 10:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-ingress-3
• 10:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-ingress-3
• 09:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the toolsbeta cluster
• 09:59 taavi@cloudcumin1001: Added a new k8s ingress toolsbeta-test-k8s-ingress-7.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 09:52 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 09:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 09:50 taavi@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-4.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 09:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 09:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-8
• 09:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-8
• 09:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-7
• 09:38 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-7

2024-02-12

• 10:49 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 10:48 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 10:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:32 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-02-09

• 10:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component image-config
• 10:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component image-config

2024-02-08

• 15:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-test-k8s-worker-nfs-3.toolsbeta.eqiad1.wikimedia.cloud
• 15:18 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-test-k8s-worker-nfs-3.toolsbeta.eqiad1.wikimedia.cloud
• 15:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-test-k8s-worker-nfs-2.toolsbeta.eqiad1.wikimedia.cloud
• 15:15 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-test-k8s-worker-nfs-2.toolsbeta.eqiad1.wikimedia.cloud
• 11:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 11:30 taavi@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-3.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 11:22 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 11:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-6
• 11:06 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-6
• 11:05 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host toolsbeat-test-k8s-worker-6
• 11:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeat-test-k8s-worker-6
• 11:01 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the toolsbeta cluster
• 11:01 taavi@cloudcumin1001: Added a new k8s worker-nfs toolsbeta-test-k8s-worker-nfs-2.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 10:53 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 10:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host toolsbeta-test-k8s-worker-10
• 10:49 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host toolsbeta-test-k8s-worker-10

2024-02-06

• 11:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 11:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:58 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all nodes (T356507)
• 10:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all nodes (T356507)

2024-02-05

• 09:55 arturo: grant myself member and admin privileges

2024-01-31

• 13:50 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 13:49 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-01-29

• 13:06 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on toolsbeta-mail-2.toolsbeta.eqiad1.wikimedia.cloud
• 13:04 wmbot~taavi@runko: START - Cookbook wmcs.vps.refresh_puppet_certs on toolsbeta-mail-2.toolsbeta.eqiad1.wikimedia.cloud

2024-01-26

• 10:59 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the toolsbeta cluster
• 10:59 wmbot~taavi@runko: Added a new k8s control toolsbeta-test-k8s-control-7.toolsbeta.eqiad1.wikimedia.cloud to the cluster
• 10:47 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster
• 10:43 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a control role in the toolsbeta cluster
• 10:42 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the toolsbeta cluster

2024-01-25

• 12:30 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the toolsbeta cluster
• 12:30 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 12:28 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the toolsbeta cluster
• 12:27 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 12:24 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the toolsbeta cluster
• 12:24 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the toolsbeta cluster
• 11:01 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 11:00 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-24

• 11:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the toolsbeta cluster

2024-01-23

• 19:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 19:10 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 19:09 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component wmcs-k8s-metrics
• 19:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 10:07 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 10:06 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-19

• 15:38 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 15:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 12:08 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 12:08 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-01-17

• 14:28 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 14:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-12

• 09:22 taavi: upgrade prometheus on toolsbeta-prometheus-1

2024-01-11

• 17:27 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 17:26 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 17:07 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 17:07 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 15:10 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 15:10 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-09

• 17:18 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 17:17 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-08

• 10:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 10:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-01-05

• 14:42 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 14:42 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 11:50 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 11:49 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 11:11 wm-bot2: dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 11:11 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2023-12-26

• 19:15 dhinus: hard reboot toolsbeta-bastion-6 as it's unreachable

2023-12-20

• 18:51 wm-bot2: fran@wmf3169 END (FAIL) - Cookbook wmcs.openstack.quota_increase (exit_code=99)
• 18:51 wm-bot2: fran@wmf3169 START - Cookbook wmcs.openstack.quota_increase
• 18:47 wm-bot2: fran@wmf3169 END (FAIL) - Cookbook wmcs.openstack.quota_increase (exit_code=99)
• 18:47 wm-bot2: fran@wmf3169 START - Cookbook wmcs.openstack.quota_increase
• 18:47 wm-bot2: fran@wmf3169 END (FAIL) - Cookbook wmcs.openstack.quota_increase (exit_code=99)
• 18:47 wm-bot2: fran@wmf3169 START - Cookbook wmcs.openstack.quota_increase
• 18:47 wm-bot2: fran@wmf3169 END (FAIL) - Cookbook wmcs.openstack.quota_increase (exit_code=99)
• 18:47 wm-bot2: fran@wmf3169 START - Cookbook wmcs.openstack.quota_increase

2023-12-15

• 13:18 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api (T341067)
• 13:17 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T341067)

2023-12-13

• 16:23 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.scale_grid_exec (exit_code=97)
• 16:23 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.scale_grid_exec
• 14:13 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api (T352774)
• 14:12 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T352774)
• 14:12 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder (T352774)
• 14:12 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder (T352774)
• 13:27 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api (T338142)
• 13:26 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T338142)
• 10:44 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-admission (T338142)
• 10:43 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-admission (T338142)
• 09:47 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 09:47 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2023-12-12

• 12:13 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api (T352774)
• 12:12 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T352774)

2023-12-11

• 19:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 19:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 15:25 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api (T352774)
• 15:24 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T352774)
• 15:23 wm-bot2: dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api (T352774)
• 15:22 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T352774)
• 13:36 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api (T352774)
• 13:35 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api (T352774)
• 13:32 dcaro: rebooted the bastion-6, did not seem to have network and was failing to mount nfs
• 13:26 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 13:25 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 13:25 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 13:23 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 13:23 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-admission (T352774)
• 13:22 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-admission (T352774)

2023-12-07

• 14:46 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 14:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2023-12-05

• 21:08 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
• 21:07 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors
• 21:07 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
• 21:07 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors
• 17:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
• 17:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors
• 09:10 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 09:07 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console

2023-12-04

• 09:03 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 09:03 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2023-12-01

• 15:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
• 15:49 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2023-11-23

• 10:35 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 10:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2023-11-22

• 11:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 11:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 10:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 09:28 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 09:27 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2023-11-20

• 15:00 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 15:00 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
• 13:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 13:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2023-11-17

• 15:03 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=97) for all nodes
• 15:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all nodes
• 14:57 taavi@cloudcumin2001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 14:57 taavi@cloudcumin2001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 14:56 taavi@cloudcumin2001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 14:56 taavi@cloudcumin2001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2023-11-09

• 15:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 15:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 10:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 10:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2023-11-01

• 09:06 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=99)
• 09:06 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2023-10-30

• 14:02 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 14:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics

2023-10-27

• 09:41 dcaro: resizing toolsbeta-prometheus-1 to 4 cores, 8Gram
• 09:21 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 09:21 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console
• 09:18 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
• 09:11 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.openstack.cloudvirt.vm_console

2023-10-26

• 09:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 09:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
• 09:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
• 09:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics

2023-10-25

• 11:01 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 11:01 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-k8s-ingress-6
• 11:01 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-k8s-ingress-6
• 10:28 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster
• 10:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolsbeta-test-k8s-ingress-6
• 10:27 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolsbeta-test-k8s-ingress-6
• 10:26 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a ingress role in the toolsbeta cluster
• 10:10 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the toolsbeta cluster

2023-10-23

• 15:33 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 15:33 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2023-10-20

• 10:37 blancadesal: harbor up again and upgraded from 2.5 to 2.9 (T346241)
• 10:11 dcaro: taking harbor down for upgrade (T346241)

2023-10-18

• 12:18 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 12:17 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2023-10-13

• 13:26 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 13:26 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 09:24 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 09:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 09:06 wm-bot2: dcaro@urcuchillay END (ERROR) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=97)
• 09:06 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2023-10-12

• 11:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
• 11:59 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2023-10-10

• 08:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 08:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 07:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 07:45 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2023-10-09

• 07:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 07:54 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2023-10-05

• 15:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 15:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2023-10-04

• 16:53 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 16:53 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 16:17 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 16:17 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 14:56 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 14:55 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 13:04 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
• 13:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
• 07:38 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 07:37 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2023-10-03

• 13:04 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 13:03 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 11:42 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
• 11:42 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
• 09:21 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
• 09:20 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission

2023-09-27

• 14:13 wm-bot2: dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
• 14:12 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 14:07 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
• 14:07 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
• 12:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component image-config
• 12:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component image-config

2023-09-25

• 07:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 07:18 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx

2023-09-20

• 06:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 06:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx

2023-09-19

• 15:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
• 15:11 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
• 14:48 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 14:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2023-09-15

• 12:26 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
• 12:26 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2023-09-14

• 12:10 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
• 12:09 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
• 12:05 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-emailer
• 12:05 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-emailer
• 11:59 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-admission
• 11:58 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-admission
• 11:57 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 11:56 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 10:16 dcaro: deploy bulids-api 0.0.96
• 09:17 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 09:16 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
• 08:54 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
• 08:53 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2023-09-13

• 16:41 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)
• 16:40 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)
• 12:38 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)
• 12:37 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)
• 12:37 wm-bot2: dcaro@urcuchillay END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component maintain-kubeusersNone (T341084)
• 12:37 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)
• 10:30 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusersNone (T341084)
• 10:29 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)
• 10:27 wm-bot2: dcaro@urcuchillay END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component maintain-kubeusersNone (T341084)
• 10:27 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusersNone (T341084)
• 10:06 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admissionNone
• 10:05 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admissionNone

2023-09-11

• 16:05 dcaro: deploy builds-builder (T341084)
• 11:36 dcaro: deploy kubernetes-metrics (T341084)

2023-09-06

• 08:47 arturo: switch project to new DNS recursor via horizon project hiera (T345240, T342621)

2023-09-05

• 13:30 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admissionNone (T341462)
• 13:29 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admissionNone (T341462)
• 13:29 wm-bot2: dcaro@urcuchillay END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component registry-admissionNone (T341462)
• 13:29 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admissionNone (T341462)
• 13:25 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admissionNone (T341462)
• 13:24 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admissionNone (T341462)

2023-08-31

• 15:42 wm-bot2: fran@wmf3169 END (PASS) - Cookbook wmcs.toolforge.grid.get_cluster_status (exit_code=0)
• 15:41 wm-bot2: fran@wmf3169 START - Cookbook wmcs.toolforge.grid.get_cluster_status
• 15:38 wm-bot2: fran@wmf3169 START - Cookbook wmcs.toolforge.grid.get_cluster_status
• 12:42 wm-bot2: fran@wmf3169 END (PASS) - Cookbook wmcs.toolforge.grid.get_job_logs (exit_code=0)
• 12:42 wm-bot2: fran@wmf3169 START - Cookbook wmcs.toolforge.grid.get_job_logs
• 12:41 wm-bot2: fran@wmf3169 END (PASS) - Cookbook wmcs.toolforge.grid.get_job_logs (exit_code=0)
• 09:36 wm-bot2: deployed kubernetes component api-gateway (c0faf0f) (T341462) - cookbook ran by dcaro@urcuchillay
• 08:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-7 from 1.22.17 to 1.23.17
• 08:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-7 from 1.22.17 to 1.23.17
• 08:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-5 from 1.22.17 to 1.23.17
• 08:36 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-5 from 1.22.17 to 1.23.17
• 08:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-4 from 1.22.17 to 1.23.17
• 08:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-4 from 1.22.17 to 1.23.17
• 08:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-ingress-3 from 1.22.17 to 1.23.17
• 08:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-ingress-3 from 1.22.17 to 1.23.17
• 08:28 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node toolsbeta-test-k8s-worker-8 from 1.22.17 to 1.23.17
• 08:27 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-8 from 1.22.17 to 1.23.17
• 08:25 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node toolsbeta-test-k8s-worker-8 from 1.22.17 to 1.23.17
• 08:24 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node toolsbeta-test-k8s-worker-8 from 1.22.17 to 1.23.17

2023-08-30

• 11:18 wm-bot2: toolsbeta-test-k8s-worker-9: upgraded k8s from 1.22.17 to 1.23.17 (T298005) - cookbook ran by arturo@nostromo
• 11:17 wm-bot2: toolsbeta-test-k8s-worker-9: upgrading k8s from 1.22.17 to 1.23.17 (T298005) - cookbook ran by arturo@nostromo
• 11:15 wm-bot2: toolsbeta-test-k8s-worker-9: upgrading k8s from 1.22.17 to 1.23.17 (T298005) - cookbook ran by arturo@nostromo
• 10:05 dcaro: upgrade toolforge-weld to 1.2.1 (T344155)
• 08:15 taavi: updating toolsbeta k8s cluster to 1.23 to test new cookbooks, T298005 T343869

2023-08-29

• 13:06 wm-bot2: deployed kubernetes component jobs-emailer (6f9c8cf) - cookbook ran by taavi@runko
• 13:03 wm-bot2: deployed kubernetes component jobs-api (b29193d) (T341462) - cookbook ran by dcaro@urcuchillay

2023-08-28

• 14:54 wm-bot2: deployed kubernetes component envvars-api (90055b5) (T344502) - cookbook ran by dcaro@urcuchillay

2023-08-22

• 14:29 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (27328a4) (T344668) - cookbook ran by taavi@runko

2023-08-18

• 13:40 wm-bot2: deployed kubernetes component envvars-api (06c26be) (T341462) - cookbook ran by dcaro@urcuchillay
• 12:30 wm-bot2: deployed kubernetes component builds-api (727e6a7) (T341462) - cookbook ran by dcaro@urcuchillay

2023-08-17

• 12:19 dcaro: deploy builds-api builds-api-0.0.85-20230817105952-25c2b55f

2023-08-11

• 09:06 taavi: fixed /etc/hosts on toolsbeta-nfs-2 because 'Template:Fqdn' is not a valid fqdn

2023-07-26

• 09:30 wm-bot2: deployed kubernetes component image-config (06066ba) - cookbook ran by taavi@runko

2023-07-25

• 12:59 wm-bot2: deployed kubernetes component image-config (0eb287a) - cookbook ran by taavi@runko

2023-07-20

• 14:34 arturo: deploying https://gitlab.wikimedia.org/repos/cloud/toolforge/buildservice/-/merge_requests/6 again with newer image (T342338, T321188)
• 10:48 arturo: deploying https://gitlab.wikimedia.org/repos/cloud/toolforge/buildservice/-/merge_requests/6 on toolsbeta

2023-07-18

• 10:45 arturo: redeploy jobs-emailer into k8s (T341084)

2023-07-13

• 14:33 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (75db740) - cookbook ran by taavi@runko

2023-07-12

• 12:46 arturo: deployed builds-admission 0.0.63-20230712120152-2ef80a7c (T341084)

2023-07-04

• 13:55 taavi: removed floating IP and public dns records for the harbor server

2023-07-03

• 19:08 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config.git (561b4d9) - cookbook ran by taavi@runko
• 08:57 wm-bot2: dcaro doing tests - cookbook ran by dcaro@urcuchillay

2023-06-26

• 07:49 dcaro: restarting harbor trove DB (in error status)

2023-06-21

• 11:48 dcaro: deploy bulids-api 0.2.0 (T337025)
• 11:48 dcaro: deploy bulids-api 0.2.0

2023-06-16

• 14:28 dcaro: deployed envvars-api 0.0.1
• 07:41 dcaro: deployed latest builds-api 0.1.0

2023-06-15

• 14:05 wm-bot2: cleaned up grid queue errors on toolsbeta-sgegrid-master - cookbook ran by andrew@bullseye

2023-06-08

• 11:54 dcaro: powering off toolsbeta-test-k8s-etcd-22 (T334644)

2023-06-07

• 12:47 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0ed420b) - cookbook ran by taavi@runko

2023-06-01

• 10:04 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (7e57832) (T337218) - cookbook ran by dcaro@vulcanus
• 09:16 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/buildpack-admission-controller (ef7f103) (T336130) - cookbook ran by dcaro@vulcanus
• 09:11 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (0f4076a) (T336130) - cookbook ran by dcaro@vulcanus
• 09:02 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (f1d94f7) (T336130) - cookbook ran by dcaro@vulcanus
• 08:57 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (6c6a27b) (T336130) - cookbook ran by dcaro@vulcanus
• 07:18 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (3488cfe) (T337218) - cookbook ran by dcaro@vulcanus

2023-05-26

• 12:44 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/buildpack-admission-controller (ef7f103) (T337218) - cookbook ran by dcaro@vulcanus
• 12:39 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/buildpack-admission-controller (d567670) (T337218) - cookbook ran by dcaro@vulcanus

2023-05-25

• 08:40 dcaro: releasing toolforge-weld 1.0.0 (T337218)

2023-05-24

• 12:26 dcaro: deploy latest buildservice (T335865)
• 12:26 dcaro: deploy latest buildservice (T336050)

2023-05-23

• 14:40 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (0c7b25b) - cookbook ran by fran@wmf3169

2023-05-16

• 14:45 dcaro: deploy builds-api (T336225)
• 14:43 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (1a725d0) - cookbook ran by dcaro@vulcanus
• 11:45 dcaro: release toolforge-weld 0.2.0 and toolforge-webservice 0.98

2023-05-15

• 13:31 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-builds-api (0277378) - cookbook ran by dcaro@vulcanus
• 09:22 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/volume-admission-controller (ad5b2b5) - cookbook ran by dcaro@vulcanus

2023-05-09

• 17:05 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/ingress-admission-controller (e89c581) - cookbook ran by taavi@runko
• 07:27 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 07:24 wm-bot2: Updating the tekton related images on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus

2023-05-05

• 11:33 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (87937cd) - cookbook ran by taavi@runko

2023-05-01

• 23:24 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (7199a9e) - cookbook ran by raymond@ubuntu

2023-04-30

• 14:52 wm-bot2: removed instance toolsbeta-test-k8s-etcd-19 - cookbook ran by taavi@runko
• 14:42 wm-bot2: removed instance toolsbeta-test-k8s-etcd-18 - cookbook ran by taavi@runko
• 14:33 wm-bot2: removed instance toolsbeta-test-k8s-etcd-17 - cookbook ran by taavi@runko

2023-04-19

• 16:17 wm-bot2: removed instance toolsbeta-test-k8s-etcd-21 - cookbook ran by taavi@runko
• 14:29 wm-bot2: removed instance toolsbeta-test-k8s-etcd-20 - cookbook ran by taavi@runko
• 14:09 wm-bot2: removed instance toolsbeta-test-k8s-etcd-20 - cookbook ran by taavi@runko
• 13:45 wm-bot2: removed instance toolsbeta-test-k8s-etcd-20 - cookbook ran by taavi@runko
• 13:34 wm-bot2: removed instance toolsbeta-test-k8s-etcd-20 - cookbook ran by taavi@runko
• 12:52 wm-bot2: removed instance toolsbeta-test-k8s-etcd-20 - cookbook ran by taavi@runko
• 12:32 wm-bot2: removed instance toolsbeta-test-k8s-etcd-20 - cookbook ran by taavi@runko
• 12:10 wm-bot2: removed instance toolsbeta-test-k8s-etcd-21 - cookbook ran by taavi@runko
• 12:07 wm-bot2: removed instance toolsbeta-test-k8s-etcd-22 - cookbook ran by taavi@runko

2023-04-11

• 14:13 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/volume-admission-controller.git (d878e49) - cookbook ran by dcaro@vulcanus
• 13:29 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (b65439b) - cookbook ran by arturo@nostromo
• 10:27 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-nginx (8f0bfcd) - cookbook ran by taavi@runko
• 08:59 wm-bot2: Added a new k8s worker toolsbeta-test-k8s-worker-9.toolsbeta.eqiad1.wikimedia.cloud to the cluster - cookbook ran by taavi@runko
• 08:46 wm-bot2: Adding a new k8s worker node - cookbook ran by taavi@runko
• 08:44 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/calico (c6a3e29) - cookbook ran by taavi@runko

2023-04-05

• 15:53 wm-bot2: rebooted k8s node toolsbeta-test-k8s-worker-7 - cookbook ran by arturo@nostromo
• 15:15 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (5ea5992) - cookbook ran by taavi@runko
• 15:12 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (2be9962) - cookbook ran by taavi@runko

2023-04-03

• 11:14 wm-bot2: rebooted k8s node toolsbeta-test-k8s-control-4 - cookbook ran by arturo@nostromo
• 11:13 wm-bot2: rebooted k8s node toolsbeta-test-k8s-control-5 - cookbook ran by arturo@nostromo
• 11:12 wm-bot2: rebooted k8s node toolsbeta-test-k8s-control-6 - cookbook ran by arturo@nostromo
• 11:11 wm-bot2: rebooted k8s node toolsbeta-test-k8s-ingress-3 - cookbook ran by arturo@nostromo
• 11:10 wm-bot2: rebooted k8s node toolsbeta-test-k8s-ingress-4 - cookbook ran by arturo@nostromo
• 11:08 wm-bot2: rebooted k8s node toolsbeta-test-k8s-ingress-5 - cookbook ran by arturo@nostromo
• 11:07 wm-bot2: rebooted k8s node toolsbeta-test-k8s-worker-6 - cookbook ran by arturo@nostromo
• 11:05 wm-bot2: rebooted k8s node toolsbeta-test-k8s-worker-7 - cookbook ran by arturo@nostromo
• 11:03 wm-bot2: rebooted k8s node toolsbeta-test-k8s-worker-8 - cookbook ran by arturo@nostromo
• 11:01 wm-bot2: rebooting the whole toolsbeta k8s cluster (9 nodes) - cookbook ran by arturo@nostromo
• 11:00 wm-bot2: rebooted k8s node toolsbeta-test-k8s-worker-7 - cookbook ran by arturo@nostromo
• 10:59 wm-bot2: rebooted k8s node toolsbeta-test-k8s-control-5 - cookbook ran by arturo@nostromo
• 10:26 wm-bot2: rebooted k8s node toolsbeta-test-k8s-worker-7 - cookbook ran by arturo@nostromo
• 10:24 wm-bot2: rebooted k8s node toolsbeta-test-k8s-control-5 - cookbook ran by arturo@nostromo
• 10:22 wm-bot2: rebooted k8s node toolsbeta-test-k8s-control-5 - cookbook ran by arturo@nostromo

2023-03-19

• 09:32 wm-bot2: cleaned up grid queue errors on toolsbeta-sgegrid-master - cookbook ran by taavi@runko

2023-03-14

• 10:39 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (b70adc1) - cookbook ran by sstefanova@Slavinas-MBP-W.local
• 10:23 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (7d4afeb) - cookbook ran by sstefanova@Slavinas-MBP-W.local

2023-03-13

• 09:27 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (f90bd8f) - cookbook ran by dcaro@vulcanus

2023-03-10

• 16:35 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (8b42b15) - cookbook ran by taavi@runko

2023-03-09

• 10:08 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (53e7f81) - cookbook ran by taavi@runko

2023-03-07

• 11:09 taavi: upgrading kubernetes to 1.22 T286856

2023-03-06

• 12:48 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (6688477) - cookbook ran by taavi@runko
• 12:45 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (21fef22) - cookbook ran by taavi@runko
• 12:36 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/registry-admission-webhook (98ce17f) - cookbook ran by taavi@runko
• 12:00 arturo: delete calico deployment, and try loading it again for https://gitlab.wikimedia.org/repos/cloud/toolforge/calico/-/merge_requests/1

2023-03-05

• 15:41 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/labs/tools/maintain-kubeusers (3e04025) - cookbook ran by taavi@runko

2023-03-02

• 11:31 arturo: aborrero@toolsbeta-test-k8s-control-4:~$ sudo -i kubectl apply -f /etc/kubernetes/toolforge-tool-roles.yaml (https://gerrit.wikimedia.org/r/c/operations/puppet/+/889836)

2023-03-01

• 13:15 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (13eda9d) - cookbook ran by taavi@runko

2023-02-28

• 17:18 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway (9252af7) - cookbook ran by taavi@runko
• 17:03 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (e46da83) - cookbook ran by taavi@runko
• 14:11 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (f90bd8f) - cookbook ran by dcaro@vulcanus

2023-02-23

• 16:37 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway (efb60b3) - cookbook ran by taavi@runko
• 16:30 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway (4e8645a) - cookbook ran by taavi@runko

2023-02-17

• 11:27 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (eeeea4c) - cookbook ran by arturo@endurance
• 11:17 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (7729b18) (T254636) - cookbook ran by arturo@endurance

2023-02-16

• 16:01 wm-bot2: renewed kubeadm certs on toolsbeta-test-k8s-control-6 - cookbook ran by arturo@nostromo
• 15:58 wm-bot2: renewed kubeadm certs on toolsbeta-test-k8s-control-5 - cookbook ran by arturo@nostromo
• 15:55 wm-bot2: renewed kubeadm certs on toolsbeta-test-k8s-control-4 - cookbook ran by arturo@nostromo
• 15:28 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/cert-manager (d71994e) - cookbook ran by arturo@nostromo
• 13:47 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/ingress-admission-controller (7191997) - cookbook ran by taavi@runko
• 10:32 arturo: aborrero@toolsbeta-test-k8s-control-4:~$ sudo -i kubectl apply -f /etc/kubernetes/psp/base-pod-security-policies.yaml

2023-02-15

• 09:30 wm-bot2: cleaned up grid queue errors on toolsbeta-sgegrid-master - cookbook ran by arturo@nostromo

2023-02-14

• 20:52 taavi: deploy cert-manager to toolsbeta T329453
• 12:02 arturo: included tools-manifests 0.25 in toolsbeta-buster aptly repo (T329611, T329467, T244809)

2023-02-13

• 15:03 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (13d87c4) - cookbook ran by taavi@runko
• 13:55 wm-bot2: drained, depooled and removed worker toolsbeta-test-k8s-worker-5 - cookbook ran by arturo@nostromo
• 13:46 wm-bot2: Depooled and removed worker toolsbeta-test-k8s-worker-4.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by arturo@nostromo
• 13:46 wm-bot2: Drained node toolsbeta-test-k8s-worker-4 - cookbook ran by arturo@nostromo
• 13:46 wm-bot2: Draining node toolsbeta-test-k8s-worker-4... - cookbook ran by arturo@nostromo
• 13:45 wm-bot2: Depooling and removing worker , will pick the oldest - cookbook ran by arturo@nostromo
• 13:31 wm-bot2: Depooling and removing worker , will pick the oldest - cookbook ran by arturo@nostromo
• 13:30 wm-bot2: Depooling and removing worker , will pick the oldest - cookbook ran by arturo@nostromo
• 13:15 arturo: cordoned & drained k8s workers 4 to 7 to force workload to relocate to 8 (T329378)
• 12:35 wm-bot2: Added a new k8s worker toolsbeta-test-k8s-worker-8.toolsbeta.eqiad1.wikimedia.cloud to the worker pool - cookbook ran by arturo@nostromo
• 12:24 wm-bot2: Adding a new k8s worker node - cookbook ran by arturo@nostromo

2023-02-10

• 16:14 wm-bot2: Adding a new k8s worker node - cookbook ran by arturo@nostromo

2023-02-01

• 15:41 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (372037f) - cookbook ran by taavi@runko

2023-01-26

• 14:33 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (307f302) - cookbook ran by taavi@runko

2023-01-23

• 11:26 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (d5ae229) (T311918) - cookbook ran by taavi@runko

2023-01-20

• 15:58 wm-bot2: renewed kubeadm certs on toolsbeta-test-k8s-control-6 - cookbook ran by arturo@nostromo
• 15:56 wm-bot2: renewed kubeadm certs on toolsbeta-test-k8s-control-5 - cookbook ran by arturo@nostromo
• 15:54 wm-bot2: renewed kubeadm certs on toolsbeta-test-k8s-control-4 - cookbook ran by arturo@nostromo

2023-01-19

• 11:46 arturo: `aborrero@toolsbeta-test-k8s-control-4:~$ sudo -i kubectl delete clusterrolebinding jobs-api-psp` (cleanup unused stuff)

2023-01-18

• 15:36 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0ad4c66) - cookbook ran by arturo@nostromo

2023-01-17

• 13:56 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (8cf38a1) - cookbook ran by arturo@endurance
• 13:46 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0d0a882) - cookbook ran by arturo@endurance
• 13:45 arturo: add login.toolsbeta.wmflabs.org DNS record as CNAME to toolsbeta-sgebastion-05.toolsbeta.eqiad1.wikimedia.cloud

2023-01-10

• 11:53 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (8e0a2f9) - cookbook ran by arturo@endurance
• 10:42 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0243967) - cookbook ran by arturo@endurance

2022-12-09

• 08:45 dcaro: manually started puppetdb after killed by oom (T324812)

2022-11-30

• 10:37 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (bc3529d) - cookbook ran by arturo@nostromo

2022-11-29

• 12:39 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (864171a) - cookbook ran by taavi@runko
• 12:22 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (a8b6e17) - cookbook ran by taavi@runko
• 09:54 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (9528ed3) - cookbook ran by taavi@runko

2022-11-28

• 18:39 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (ec5c82b) - cookbook ran by taavi@runko
• 18:36 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/image-config (5394a34) - cookbook ran by taavi@runko

2022-11-15

• 12:40 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org/test-raymond - cookbook ran by raymond@ubuntu
• 11:36 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org/test-raymond - cookbook ran by raymond@ubuntu

2022-11-14

• 20:05 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org/test-raymond - cookbook ran by raymond@ubuntu
• 19:58 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org/test-raymond - cookbook ran by raymond@ubuntu
• 14:14 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by fran@wmf3169
• 14:14 wm-bot2: Updating the lifecycle image on docker-registry.tools.wmflabs.org - cookbook ran by fran@wmf3169
• 14:14 wm-bot2: Updating the bash image on docker-registry.tools.wmflabs.org - cookbook ran by fran@wmf3169
• 14:12 wm-bot2: Updating the tekton related images on docker-registry.tools.wmflabs.org - cookbook ran by fran@wmf3169

2022-11-07

• 13:32 wm-bot2: deployed kubernetes component https://github.com/toolforge/buildpack-admission-controller (b4e912e) - cookbook ran by fran@wmf3169

2022-11-04

• 12:24 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (d464be4) (T304900) - cookbook ran by arturo@nostromo

2022-11-01

• 12:42 taavi: remove labstore1006/7 from acme-chief-1 fstab and reboot

2022-10-24

• 16:42 wm-bot2: rebooted buster webgen grid workers - cookbook ran by andrew@bullseye
• 16:29 wm-bot2: rebooting buster webgen grid workers - cookbook ran by andrew@bullseye
• 14:54 wm-bot2: Increased quotas by 30 gigabytes - cookbook ran by dcaro@vulcanus

2022-10-18

• 10:24 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer (64385e9) (T320405) - cookbook ran by arturo@nostromo

2022-10-17

• 14:37 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:37 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:36 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:35 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:28 wm-bot2: Updating the lifecycle image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:27 wm-bot2: Updating the bash image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:25 wm-bot2: Updating the tekton related images on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:17 wm-bot2: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:16 wm-bot2: Updating the lifecycle image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:16 wm-bot2: Updating the bash image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus
• 14:14 wm-bot2: Updating the tekton related images on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus

2022-10-14

• 07:53 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (0cc020e) - cookbook ran by taavi@runko

2022-10-12

• 10:29 dcaro: deploying new registry-admission controller

2022-10-10

• 08:41 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (afa90ed) (T320284) - cookbook ran by taavi@runko

2022-09-28

• 09:48 arturo: manually starting gridengine-master.service on toolsbeta-sgegrid-master (T318788)

2022-09-27

• 14:23 arturo: briefly livehacking puppetmaster

2022-08-24

• 11:55 wm-bot2: deployed kubernetes component https://gitlab.wikimedia.org/repos/cloud/toolforge/ingress-nginx (7d0e951) - cookbook ran by taavi@runko

2022-08-12

• 07:24 dcaro_away: started postgresql on puppetdb-02, might have crashed during the ceph issues, now puppet runs on toolsbeta work again

2022-08-03

• 15:46 dhinus: recreated jobs-api pods to pick up new ConfigMap
• 14:51 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (c47ac41) - cookbook ran by fran@MacBook-Pro.station

2022-08-01

• 14:01 taavi: unbreak acme-chief after keystone communication issues

2022-07-19

• 15:45 taavi: deploying and testing maintain-kubeusers updates

2022-06-28

• 15:23 wm-bot2: Adding a new k8s worker node - cookbook ran by taavi@runko

2022-06-24

• 07:01 wm-bot2: removing grid node toolsbeta-sgewebgrid-lighttpd-0901.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko
• 06:59 wm-bot2: removing grid node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko
• 06:57 wm-bot2: removing grid node toolsbeta-sgeexec-0902.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko
• 06:55 wm-bot2: removing grid node toolsbeta-sgeexec-0901.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by taavi@runko

2022-06-19

• 16:28 taavi: restart OOM'd puppetdb on toolsbeta-puppetdb-02

2022-06-03

• 13:17 bd808: publish tools-webservice 0.86 (T309821)
• 05:25 wm-bot2: rebooted buster weblight grid workers - cookbook ran by taavi@runko
• 05:20 wm-bot2: rebooting buster weblight grid workers - cookbook ran by taavi@runko
• 05:20 wm-bot2: rebooting stretch weblight grid workers - cookbook ran by taavi@runko

2022-05-30

• 13:42 taavi: run grid-configurator to remove stale config for some removed nodes

2022-05-26

• 15:38 wm-bot2: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (e6fa299) - cookbook ran by taavi@runko

2022-04-20

• 07:53 wm-bot: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (8f37a04) (T305592) - cookbook ran by taavi@runko

2022-04-15

• 13:26 taavi: shutdown toolsbeta-services-01, not exactly sure what it does and it has no roles applied T306100

2022-04-11

• 14:47 dcaro: deploying custom version of the regitsry admission hook

2022-04-08

• 10:45 arturo: disabled debug mode on the k8s jobs-emailer component

2022-04-05

• 07:43 wm-bot: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (d7d3463) - cookbook ran by arturo@nostromo
• 07:21 arturo: deploying toolforge-jobs-framework-cli v7

2022-04-04

• 16:58 wm-bot: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-api (cbcfc47) - cookbook ran by arturo@nostromo
• 09:28 arturo: deployed toolforge-jobs-framework-cli v6 into aptly and installed it on buster bastions

2022-03-25

• 11:31 dcaro: All alerting VMs rebooted, checking that everything is "working" (T304672)
• 10:55 dcaro: force restarting all the other nfs-bound VMs one by one (T304672)
• 10:43 dcaro: restarting the sge-shadow (T304672)
• 10:32 dcaro: restarting the sge-master (T304672)

2022-03-16

• 15:23 taavi: deploying https://gerrit.wikimedia.org/r/c/cloud/toolforge/volume-admission-controller/+/737171/ as a T292238 test to toolsbeta

2022-03-15

• 17:55 wm-bot: deployed kubernetes component https://gerrit.wikimedia.org/r/cloud/toolforge/jobs-framework-emailer (084ee51) - cookbook ran by arturo@nostromo

2022-03-14

• 16:14 wm-bot: Updating the distroless/base image on docker-registry.tools.wmflabs.org - cookbook ran by dcaro@vulcanus

2022-03-11

• 15:55 dcaro: added provisional toolforg cli package to toolsbeta buster repo (T299026)
• 15:11 dcaro: added tekton cli package to toolsbeta repos (T299026)
• 15:02 arturo: deploy jobs-framework-emailer 9470a5f (T286135)
• 11:59 arturo: deploy jobs-framework-emailer d60ffd6 (T286135)

2022-03-08

• 08:20 taavi: reboot toolsbeta-cumin-1 for kernel updates

2022-03-07

• 15:44 dcaro: Deployed buildpack-admission-controller with the latest code (T297090)

2022-02-17

• 08:16 taavi: made toolsbeta-puppetmaster-04 its own client to fix `puppet node deactivate` puppetdb access

2022-02-08

• 13:04 arturo: livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/760933 (T284767)
• 12:19 arturo: created puppet prefix `toolsbeta-sgecron` with proper hiera/roles
• 12:16 arturo: created VM toolsbeta-sgecron-02 (T284767)

2022-02-04

• 18:53 taavi: upgrading to kubernetes 1.21 T282942

2022-01-28

• 16:28 wm-bot: trying to join node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the grid cluster in toolsbeta. - cookbook ran by arturo@nostromo

2022-01-25

• 11:45 wm-bot: reconfiguring the grid by using grid-configurator - cookbook ran by arturo@nostromo

2022-01-20

• 12:35 wm-bot: removing grid node toolsbeta-sgeexec-1003 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo
• 12:34 wm-bot: removing grid node toolsbeta-sgeexec-1004 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo

2022-01-19

• 14:11 arturo: craeted 'automated-toolforge-tests' tool account following https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/Toolsbeta#create_a_tool_account_in_toolsbeta

2022-01-18

• 15:56 wm-bot: removing grid node toolsbeta-sgewebgrid-generic-0901 (depool/drain, remove VM and reconfigure grid) - cookbook ran by andrew@buster
• 15:30 andrewbogott: switching scratch mount over to the cloud-hosted service with git fetch https://gerrit.wikimedia.org/r/operations/puppet refs/changes/43/754043/1 && git cherry-pick FETCH_HEAD
• 09:46 arturo: creating VM toolsbeta-sgebastion-05, deleting toolsbeta-bastion-05 (wrong prefix)

2022-01-17

• 18:09 wm-bot: pooled grid node toolsbeta-sgewebgen-10-1 - cookbook ran by arturo@nostromo
• 18:07 wm-bot: pooled grid node toolsbeta-sgeexec-10-5 - cookbook ran by arturo@nostromo
• 17:54 wm-bot: removing grid node toolsbeta-sgewebgen-10-4 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo
• 13:39 wm-bot: pooled grid node toolsbeta-sgeexec-10-5 - cookbook ran by arturo@nostromo

2022-01-14

• 11:56 wm-bot: removing grid node toolsbeta-sgewebgen-10-5 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo
• 11:49 wm-bot: removing grid node toolsbeta-sgeexec-10-5 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo
• 09:57 wm-bot: removing grid node toolsbeta-sgeexec-10-6.toolsbeta.eqiad1.wikimedia.cloud (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo
• 09:53 wm-bot: removing grid node toolsbeta-sgeexec-10-6.toolsbeta.eqiad1.wikimedia.org (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo
• 09:44 wm-bot: removing grid node toolsbeta-sgeweblight-10-2 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@nostromo

2022-01-12

• 12:28 wm-bot: created node toolsbeta-sgeweblight-10-1.toolsbeta.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by arturo@nostromo
• 11:27 arturo: created puppet prefix `toolsbeta-sgeweblight`, drop `toolsbeta-sgeweblig`
• 11:02 arturo: created puppet prefix 'toolsbeta-sgeweblig'
• 11:00 wm-bot: created node toolsbeta-sgeexec-10-6.toolsbeta.eqiad1.wikimedia.cloud and added it to the grid - cookbook ran by arturo@nostromo

2022-01-11

• 11:11 wm-bot: created a grid exec node toolsbeta-sgeexec-10-5.toolsbeta.eqiad1.wikimedia.cloud - cookbook ran by arturo@nostromo
• 09:20 wm-bot: reconfiguring the grid by using grid-configurator - cookbook ran by arturo@nostromo

2021-12-23

• 13:32 wm-bot: trying to join node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud to the grid cluster in toolsbeta. - cookbook ran by arturo@endurance
• 12:11 wm-bot: Added a new grid webgrid generic node toolsbeta-sgewebgen-10-4.toolsbeta.eqiad1.wikimedia.cloud to the pool - cookbook ran by arturo@endurance
• 11:58 wm-bot: node toolsbeta-sgewebgen-10-2.toolsbeta.eqiad1.wikimedia.cloud joined the grid cluster in toolsbeta. - cookbook ran by arturo@endurance
• 11:40 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance
• 11:26 wm-bot: Joining grid node toolsbeta-sgewebgen-10-2.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 11:25 wm-bot: Joining grid node toolsbeta-sgewebgen-10-2 to the toolsbeta cluster - cookbook ran by arturo@endurance
• 11:24 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance
• 10:59 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance
• 10:34 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance
• 10:31 wm-bot: reconfiguring the grid by using grid-configurator - cookbook ran by arturo@endurance

2021-12-22

• 12:02 wm-bot: reconfiguring the grid by using grid-configurator - cookbook ran by arturo@endurance
• 12:02 wm-bot: reconfiguring the grid by using grid-configurator - cookbook ran by arturo@endurance
• 12:01 wm-bot: reconfiguring the grid by using grid-configurator - cookbook ran by arturo@endurance
• 11:24 wm-bot: removing instance toolsbeta-sgewebgen-09-1 - cookbook ran by arturo@endurance
• 11:21 wm-bot: removing grid node toolsbeta-sgewebgen-09-1 (depool/drain, remove VM and reconfigure grid) - cookbook ran by arturo@endurance
• 11:19 wm-bot: depooled grid node toolsbeta-sgewebgen-10-1 - cookbook ran by arturo@endurance
• 10:42 wm-bot: depooled grid node toolsbeta-sgewebgen-10-1 - cookbook ran by arturo@endurance

2021-12-21

• 16:32 wm-bot: removing instance toolsbeta-sgewebgen-10-2 - cookbook ran by arturo@endurance
• 16:24 wm-bot: Node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud joined the grid cluster toolsbeta. - cookbook ran by arturo@endurance
• 16:24 wm-bot: Joining grid node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 12:50 wm-bot: Joining grid node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 12:07 wm-bot: Joining grid node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 12:04 wm-bot: Node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud joined the grid cluster toolsbeta. - cookbook ran by arturo@endurance
• 12:04 wm-bot: Joining grid node toolsbeta-sgewebgen-10-3.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 12:03 wm-bot: Node toolsbeta-sgewebgen-10-2.toolsbeta.eqiad1.wikimedia.cloud joined the grid cluster toolsbeta. - cookbook ran by arturo@endurance
• 12:03 wm-bot: Joining grid node toolsbeta-sgewebgen-10-2.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 11:48 wm-bot: Joining grid node toolsbeta-sgewebgen-10-2.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance
• 11:06 arturo: bump quotas, instances from 50 to 55, CPU from 100 to 150, RAM from 200GB to 250GB (T277653)

2021-12-16

• 12:46 wm-bot: Joining grid node toolsbeta-sgewebgen-10-1.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by arturo@endurance

2021-12-15

• 14:03 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance
• 13:31 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance
• 13:29 wm-bot: Adding a new grid webgrid generic node - cookbook ran by arturo@endurance

2021-12-08

• 05:15 andrewbogott: moving toolsbeta-test-k8s-etcd-17 to cloudvirt1028

2021-11-28

• 17:44 andrewbogott: moving toolsbeta-test-k8s-etcd-17 to cloudvirt1019; cloudvirt1018 (its old host) has a degraded raid which is affecting performance

2021-11-16

• 12:37 majavah: testing calico 3.21 upgrade T292698

2021-11-05

• 19:07 majavah: testing registry-admission changes

2021-10-28

• 12:48 arturo: update ingress-nginx via helm for `--watch-ingress-without-class=true`

2021-10-25

• 14:41 majavah: deploy ingress-nginx v1.0.4 to toolsbeta via helm, diff only changes the image T292771

2021-10-20

• 12:15 majavah: upload toolforge-webservice 0.78 to stretch,buster,bullsye-toolsbeta repositories

2021-10-16

• 07:47 majavah: deployed cert-manager and wave as a test for automating T292238

2021-10-14

• 15:02 wm-bot: Joining grid node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by dcaro@vulcanus
• 15:01 wm-bot: Joining grid node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by dcaro@vulcanus
• 15:00 wm-bot: Joining grid node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the toolsbeta cluster - cookbook ran by dcaro@vulcanus

2021-10-13

• 11:18 wm-bot: Added a new grid webgrid generic node toolsbeta-sgewebgen-09-1.toolsbeta.eqiad1.wikimedia.cloud to the pool (T292465) - cookbook ran by dcaro@vulcanus
• 10:19 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 10:19 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus

2021-10-12

• 16:10 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 14:52 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 14:46 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 07:05 majavah: start gridengine-master.service on toolsbeta-sgegrid-master

2021-10-11

• 15:24 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 15:00 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 10:32 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus

2021-10-07

• 14:21 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 14:06 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 13:31 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 12:55 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 12:50 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 12:50 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 08:04 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 07:58 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus

2021-10-06

• 10:36 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 10:13 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 10:08 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 10:07 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus
• 10:05 wm-bot: Adding a new grid webgrid generic node (T292465) - cookbook ran by dcaro@vulcanus

2021-10-04

• 17:07 bstorm: reboot everything T291406
• 17:06 bstorm: use cumin to edit fstab to remove old nfs mounts T291406
• 16:41 bstorm: setting mount_nfs: true on toolsbeta-mail prefix (which is the correct setting)
• 14:45 dcaro: rebooting toolsbeta-sgewebgrid-generic-0901.toolsbeta.eqiad1.wikimedia.cloud to force a fsck of the dm-0 device on boot (T290970)

2021-10-01

• 12:34 arturo: rebooting toolsbeta-sgebastion-04 (T292289)
• 12:12 arturo: experimenting with newer mono runtime on toolsbeta-sgebastion-04 (T292289)

2021-09-29

• 22:13 bstorm: ran label fix script to use new label format
• 22:12 bstorm: toollabs-webservice 0.77 deployed

2021-09-28

• 10:32 majavah: removing all podpreset objects and disabling settings.k8s.io/v1alpha1 api

2021-09-27

• 16:13 majavah: testing volume-admission fix for containers with some volumes mounted

2021-09-23

• 17:14 majavah: testing new maintain-kubeusers release T279106

2021-09-22

• 18:07 bstorm: launching toolsbeta-nfs-test-client-01 to run a "fair" test battery against T291406

2021-09-15

• 08:04 majavah: tools-manifest 0.24, T290325

2021-09-14

• 15:45 majavah: disable podpreset admission plugin in toolsbeta T279106
• 11:42 arturo: deploying jobs-framework-emailer 3045601 (T286135)
• 10:44 arturo: deploying jobs-framework-emailer 51032af (T286135)
• 10:39 arturo: deploying jobs-framework-api 16fbf51 (T286135)

2021-09-13

• 15:44 majavah: deploy volume-admission-controller in background; T279106

2021-09-09

• 17:36 bstorm: deploying a base tekton triggers setup T267374
• 16:50 majavah: enable unattended updates on toolsbeta T290494
• 16:19 arturo: 70017ec0ac root@toolsbeta-test-k8s-control-4:~# kubectl apply -f /etc/kubernetes/psp/base-pod-security-policies.yaml
• 00:26 bstorm: deleted toolsbeta-sgeexec-0902 since it had a badly screwed up /tmp

2021-09-03

• 22:34 bstorm: backfilled quotas for T286784

2021-08-30

• 23:23 bstorm: deleting toolsbeta-workflow-test T289709

2021-08-21

• 00:17 bstorm: rebooting the control plane nodes for kubernetes because it can't make things worse T289390

2021-08-20

• 23:19 bstorm: tried renewing all the certs to get certs working again in kubernetes

2021-08-12

• 16:55 bstorm: deployed updated manifest for ingress-admission
• 15:02 majavah: deploying ingress-admission-controller using v1 api T280436

2021-07-30

• 08:01 majavah: replace toolsbeta-sgeexec-1002 with -1004 for T287666

2021-07-29

• 14:08 majavah: add mdipietro as projectadmin T287287
• 13:06 majavah: rebuild toolsbeta-sgeexec-1001 as -1003 T287666

2021-07-23

• 13:31 majavah: upgrading toolsbeta to kubernetes 1.19, T280340

2021-07-22

• 15:32 arturo: re-deploying toolforge-jobs-framework-api

2021-07-21

• 11:58 arturo: deploying jobs-framework-api 07346d7 (T286108)
• 10:51 arturo: enabling TTLAfterFinished feature gate on static pod manifests on /etc/kubernetes/manifests/kube-{apiserver,controller-manager}.yaml in all 3 control nodes (T286108)
• 10:47 arturo: enabling TTLAfterFinished feature gate on kubeadm live configmap (T286108)
• 10:09 arturo: livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/705848

2021-07-20

• 21:18 bstorm: applied `login_server: true` to toolsbeta-sgecron-01 T287037
• 19:09 bstorm: upgraded version of maintain-kubeusers to the latest in master branch T285011
• 08:36 majavah: resolve merge conflicts on labs/private

2021-07-16

• 19:53 bstorm: set matchPolicy to equivalent on ingress admission controller for toolsbeta T280360
• 14:04 arturo: deployed jobs-framework-api 42b7a88 (T286132)

2021-07-15

• 15:39 arturo: deploy toolforge-jobs-framework-api git version d85d93e

2021-07-14

• 09:05 majavah: testing calico 3.18 upgrade - T280342

2021-07-12

• 11:42 majavah: rebooting toolsbeta-sgeexec-1002, nfs issues

2021-07-07

• 09:48 majavah: set dummy values for openstack ldap user/pass hiera values for disable_tool manifests to work

2021-07-01

• 17:01 majavah: updating jobs-framework-api
• 10:00 arturo: refreshed jobs-api deployment

2021-06-29

• 09:28 wm-bot: Depooled and removed worker toolsbeta-test-k8s-worker-3.toolsbeta.eqiad1.wikimedia.cloud. (T267140) - cookbook ran by dcaro@vulcanus
• 09:28 wm-bot: Drained node toolsbeta-test-k8s-worker-3. (T267140) - cookbook ran by dcaro@vulcanus
• 09:27 wm-bot: Draining node toolsbeta-test-k8s-worker-3... (T267140) - cookbook ran by dcaro@vulcanus
• 09:27 wm-bot: Depooling and removing worker , will pick the oldest. (T267140) - cookbook ran by dcaro@vulcanus
• 09:27 wm-bot: Added a new k8s worker toolsbeta-test-k8s-worker-6.toolsbeta.eqiad1.wikimedia.cloud to the worker pool - cookbook ran by dcaro@vulcanus
• 09:18 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 09:13 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 09:13 wm-bot: Depooled and removed worker toolsbeta-test-k8s-worker-2.toolsbeta.eqiad1.wikimedia.cloud. (T267140) - cookbook ran by dcaro@vulcanus
• 09:13 wm-bot: Drained node toolsbeta-test-k8s-worker-2. (T267140) - cookbook ran by dcaro@vulcanus
• 09:12 wm-bot: Draining node toolsbeta-test-k8s-worker-2... (T267140) - cookbook ran by dcaro@vulcanus
• 09:12 wm-bot: Depooling and removing worker , will pick the oldest. (T267140) - cookbook ran by dcaro@vulcanus
• 09:09 wm-bot: Added a new k8s worker toolsbeta-test-k8s-worker-5.toolsbeta.eqiad1.wikimedia.cloud to the worker pool - cookbook ran by dcaro@vulcanus
• 09:00 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 08:59 wm-bot: Depooled and removed worker toolsbeta-test-k8s-worker-1.toolsbeta.eqiad1.wikimedia.cloud. (T267140) - cookbook ran by dcaro@vulcanus
• 08:59 wm-bot: Drained node toolsbeta-test-k8s-worker-1. (T267140) - cookbook ran by dcaro@vulcanus
• 08:58 wm-bot: Draining node toolsbeta-test-k8s-worker-1... (T267140) - cookbook ran by dcaro@vulcanus
• 08:58 wm-bot: Depooling and removing worker , will pick the oldest. (T267140) - cookbook ran by dcaro@vulcanus
• 08:57 wm-bot: Draining node toolsbeta-test-k8s-worker-1... (T267140) - cookbook ran by dcaro@vulcanus
• 08:57 wm-bot: Depooling and removing worker , will pick the oldest. (T267140) - cookbook ran by dcaro@vulcanus

2021-06-28

• 14:46 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:45 wm-bot: Depooled and removed worker toolsbeta-test-k8s-worker-4.toolsbeta.eqiad1.wikimedia.cloud. - cookbook ran by dcaro@vulcanus
• 14:45 wm-bot: Drained node toolsbeta-test-k8s-worker-4. - cookbook ran by dcaro@vulcanus
• 14:45 wm-bot: Draining node toolsbeta-test-k8s-worker-4... - cookbook ran by dcaro@vulcanus
• 14:45 wm-bot: Depooling and removing worker toolsbeta-test-k8s-worker-4.toolsbeta.eqiad1.wikimedia.cloud. - cookbook ran by dcaro@vulcanus
• 13:23 wm-bot: Draining node toolsbeta-test-k8s-worker-4... - cookbook ran by dcaro@vulcanus
• 13:22 wm-bot: Draining node toolsbeta-test-k8s-worker-4... - cookbook ran by dcaro@vulcanus
• 13:16 wm-bot: Draining node toolsbeta-test-k8s-worker-4.toolsbeta.eqiad1.wikimedia.cloud... - cookbook ran by dcaro@vulcanus
• 11:30 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:25 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:23 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:21 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:16 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:15 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:12 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:06 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 11:06 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:54 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:53 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:44 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:27 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:27 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:16 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:15 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 10:11 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 09:56 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 09:16 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 08:51 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 08:35 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus

2021-06-25

• 15:27 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:26 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:21 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:19 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:17 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:15 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:08 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:07 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:03 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:02 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:00 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:59 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:56 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:52 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:50 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:45 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:19 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:18 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 13:57 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 13:56 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 13:55 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 13:50 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 12:50 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 12:26 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 12:26 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus

2021-06-24

• 15:52 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:35 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:35 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:33 dcaro: created flavor g3.cores4.ram8.disk20.ephem40 for the k8s workers
• 15:10 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 15:09 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:59 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:35 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:31 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:28 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:24 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus
• 14:13 wm-bot: Adding a new k8s worker node - cookbook ran by dcaro@vulcanus

2021-06-22

• 18:24 majavah: rolling out kubernetes patch release 1.18.20, cluster is currently at 1.18.18

2021-06-17

• 11:44 majavah: toolsbeta-puppetdb-02: stop puppetdb to free up its ram usage, start postgres process, start puppetdb up again

2021-06-16

• 15:53 majavah: add default security group rule allowing prometheus01.metricsinfra to connect to node-exporter port 9100

2021-06-15

• 16:10 majavah: set toolsbeta-bastion-05 as grid submit host

2021-06-14

• 21:29 bstorm: deploy package with the staged patch to switch away from os.execv to QA in toolsbeta as toollabs-webservice version 0.75 T282975
• 10:19 arturo: deploying toolforge jobs-framework-api in kubernetes (just a test) (T283238)

2021-06-12

• 14:42 majavah: sync hiera key prometheus_nodes to match tools

2021-06-11

• 15:25 majavah: undeploy nginx-ingress-jobs from kubernetes
• 14:54 majavah: generate and add own root key to passwords::root::extra_keys

2021-06-08

• 15:11 majavah: updating k8s worker nodes to 1.18 T280299
• 15:02 majavah: continuing to update k8s ingress nodes T280299
• 14:57 majavah: continuing to update rest of k8s control nodes T280299
• 14:42 majavah: remove toolsbeta-test-k8s-etcd-[15,16] from kubernetes, instances do not exist, likely leftovers from local storage work
• 14:08 majavah: update toolsbeta-test-k8s-control-4 to kubernetes 1.18 T280299

2021-06-03

• 16:55 majavah: renew ingress-admission-controller certificates T280301
• 16:49 majavah: renew registry-admission-webhook certificates T280301

2021-05-25

• 17:14 andrewbogott: deleting old ingress controllers toolsbeta-test-k8s-ingress-1 and toolsbeta-test-k8s-ingress-2
• 17:13 andrewbogott: created two new ingress nodes, toolsbeta-test-k8s-ingress-4 and toolsbeta-test-k8s-ingress-5
• 15:09 dcaro: turning off VM toolsbeta-test-k8s-etcd-14 to be able to reboot cloudvirt1020

2021-05-24

• 19:40 andrewbogott: replacing existing etcd nodes with localdisk nodes

2021-05-19

• 11:35 Majavah: testing https://gerrit.wikimedia.org/r/c/operations/puppet/+/692875/
• 06:51 Majavah: depool toolsbeta-test-k8s-ingress-1

2021-05-15

• 07:52 Majavah: set profile::wmcs::kubeadm::control::apiserver_cert_alternative_names hiera key and adjust config map T262562

2021-05-14

• 11:22 arturo: allowed VIP address from the new port 172.16.3.26 into the ports of toolsbeta-redis-[1-3] (T153810)
• 11:16 arturo: aborrero@cloudcontrol1005:~ $ sudo wmcs-openstack --os-project-id=toolsbeta port create --network lan-flat-cloudinstances2b toolsbeta-redis-vip (T153810)

2021-05-13

• 08:07 Majavah: creating toolsbeta-redis-[1-3] as g3.cores1.ram2.disk20 to experiment with redis-sentinel / T153810

2021-05-10

• 19:42 bstorm: setting profile::wmcs::kubeadm::docker_vol: false on ingress nodes
• 17:43 Majavah: testing https://gerrit.wikimedia.org/r/c/operations/puppet/+/688361 in toolsbeta T264221
• 11:50 Majavah: testing ingress-nginx update https://gerrit.wikimedia.org/r/c/operations/puppet/+/685715 on toolsbeta T264221

2021-05-08

• 10:42 Majavah: create new ingress node toolsbeta-k8s-ingress-3 T264221

2021-05-07

• 17:00 bstorm: deleted "toolsbeta-test-k8s-haproxy-2", "toolsbeta-test-k8s-haproxy-1" when the dns caches finally dropped T282227
• 16:30 bstorm: recreated k8s.toolsbeta.eqiad1.wikimedia.cloud. as a CNAME to k8s.svc.toolsbeta.eqiad1.wikimedia.cloud. T282227
• 16:16 Majavah: create record k8s.svc.toolsbeta.eqiad1.wikimedia.cloud. pointing to haproxy vip T282227
• 14:20 Majavah: cherry pick https://gerrit.wikimedia.org/r/c/operations/puppet/+/686607/
• 09:44 arturo: `sudo wmcs-openstack --os-project-id=toolsbeta port create --network lan-flat-cloudinstances2b toolsbeta-k8s-haproxy-keepalived-vip`
• 08:19 Majavah: rebuild toolsbeta-test-k8s-haproxy-[12] without nfs

2021-05-05

• 16:25 Majavah: add self to sudo policy `roots`
• 16:07 arturo: grant `taavi` projectadmin (Majavah)

2021-05-04

• 10:47 arturo: rebase & resolve merge conflicts in labs/private.git

2021-05-03

• 13:23 arturo: livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/684032 (T278109)

2021-04-29

• 18:10 bstorm: added and removed an etcd node

2021-04-23

• 17:24 bstorm: rebooting toolsbeta-test-k8s-control-6 because it was "notready" for some reason

2021-04-20

• 19:01 bstorm: updated the maintain-kubeusers:beta image to https://gerrit.wikimedia.org/r/c/labs/tools/maintain-kubeusers/+/680244

2021-04-13

• 16:41 arturo: create VM toolsbeta-sgeexec-1002 (T277653)
• 15:44 arturo: delete VMs toolsbeta-sgeexec-0903 and toolsbeta-buster-sgeexec-01 (no longer useful)
• 15:36 arturo: created VM toolsbeta-sgeexec-0903 (buster) (T277653)
• 15:31 arturo: live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/678043/ (T277653)

2021-04-08

• 18:27 bstorm: cleaned up the deprecated entries in /data/project/.system_sge/gridengine/etc/submithosts for toolsbeta-sgegrid-master and toolsbeta-sgegrid-shadow using the old fqdns T277653

2021-04-06

• 13:11 dcaro: Removing etcd member toolsbeta-test-k8s-etcd-7.tools.eqiad1.wikimedia.cloud to get an odd number (T267082)

2021-04-01

• 15:17 dcaro: etcd cluster shrunk 3 members (using wmcs.toolforge.remove_etcd_node cookbook)
• 14:54 dcaro: shrinking etcd cluster to 3 members, cleaning up automation runs

2021-03-31

• 18:22 bstorm: redeploy ingress-admission controller with `kubectl apply -k deploys/toolsbeta` from the repo T275478

2021-03-24

• 12:17 arturo: attach the `toolsbeta-docker-registry-data` volume to the `toolsbeta-docker-registry-02` VM
• 11:41 arturo: created VM toolsbeta-docker-registry-02 as Debian buster (T278303)
• 11:34 arturo: attached cinder volume `toolsbeta-docker-registry-data` as /dev/vdb on toolsbeta-docker-registry-01
• 11:23 arturo: created 2G cinder volume `toolsbeta-docker-registry-data` (T278303)

2021-03-23

• 11:22 arturo: drop and build again the VM toolsbeta-sgregrid-master (T277653)
• 11:07 arturo: drop and build again the VM toolsbeta-sgregrid-shadow (T277653)

2021-03-18

• 18:55 bstorm: set profile::toolforge::infrastructure across the entire project with login_server set on the bastion prefix
• 18:50 arturo: deleting VMs toolsbeta-paws-worker-1001 toolsbeta-paws-worker-1002 toolsbeta-paws-master-01 (testing for PAWS should happen in the paws project)
• 18:49 arturo: deleting VM toolsbeta-workflow-test, no longer useful
• 18:44 arturo: replacing toolsbeta-sgegrid-master with a Debian Buster VM (T277653)
• 16:24 arturo: live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/672456
• 12:53 arturo: create anti-affinity server group toolsbeta-sgegrid-master-shadow
• 12:51 arturo: rebuild toolsbeta-sgegrid-shadow instance as debian buster (T277653)
• 12:50 arturo: added puppet prefix `toolsbeta-sgegrid-shadow`, migrate puppet config from VM to here
• 12:48 arturo: destroy VM toolsbeta-buster-gridmaster (no longer useful) T277653
• 12:47 arturo: delete puppet prefix `toolsbeta-buster-grirdmaster` (no longer useful) T277653

2021-03-17

• 12:39 arturo: created VM toolsbeta-buster-gridmaster (T277653)
• 12:38 arturo: created puppet prefix 'toolsbeta-buster-gridmaster' (T277653)
• 12:00 arturo: create VM toolsbeta-buster-sgeexec-01 (T277653)
• 11:56 arturo: created puppet prefix 'toolsbeta-buster-sgeexec' (T277653)
• 10:34 arturo: re-create toolsbeta-bastion-05 (T275865)

2021-03-16

• 12:32 arturo: added packages jobutils / misctools v1.41 to {stretch,buster}-toolsbeta aptly repository in tools-sge-services-03

2021-03-11

• 12:33 arturo: livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/667144 for T275865

2021-03-10

• 16:48 arturo: briefly stopping VM toolsbeta-test-k8s-etcd-8 to migrate hypervisor

2021-02-26

• 20:39 andrewbogott: rebooting all hosts
• 15:35 dcaro: removed toolsbeta-test-k8s-etcd-9 with depool from kubeadmin/etcd (T274497)
• 11:46 arturo: `openstack server create --os-project-id toolsbeta --image debian-10.0-buster --flavor g2.cores2.ram4.disk40 --network lan-flat-cloudinstances2b --property description='buster bastion test' toolsbeta-bastion-05` (T275865)
• 11:39 arturo: created puppet prefix 'toolsbeta-bastion' to hold new configuration for buster-based bastions (T275865)
• 09:09 dcaro: Playing around with cookbooks by adding/removing etcd nodes, etcd might missbehave from time to time (T274497)

2021-02-19

• 12:42 arturo: deploying new version of the ingress admission controller
• 11:46 arturo: merged https://gerrit.wikimedia.org/r/c/operations/puppet/+/662941 (T274139) which should only affect toolsbeta
• 10:27 arturo: create DNS record `jobs.svc.toolsbeta.eqiad1.wikimedia.cloud` with CNAME to `k8s.toolsbeta.eqiad1.wikimedia.cloud` (T274139)
• 10:25 arturo: create DNS zone `svc.toolsbeta.eqiad1.wikimedia.cloud` (T274139)

2021-02-10

• 12:34 arturo: live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/662941 (T274139)
• 12:23 arturo: add `webserver` security group to toolsbeta-proxy-3 and -4
• 12:20 arturo: fix A record for `toolsbeta.wmflabs.org`, point it to 172.16.1.150 (toolsbeta-proxy-3), it was previously pointing to an old IP address

2021-02-08

• 11:48 arturo: trying to introduce TLS support in the front proxy T274123

2021-02-05

• 00:36 bstorm: updated jobutils and miscutils to 1.40 in aptly for toolsbeta testing

2021-01-21

• 15:29 bstorm: pushed the maintain-kubeusers:beta tag with the new code to the docker repo T271847

2021-01-13

• 14:10 dcaro: dcaro doing puppet tests, puppet runs might break
• 10:07 arturo: allocate floating IP 185.15.56.84, and use it for docker-registry.toolsbeta.wmflabs.org (instance toolsbeta-docker-registry-01) (T271867)
• 10:05 arturo: release and delete floating IP 185.15.56.242 (docker-registry.toolsbeta.wmflabs.org) (T271867)

2020-12-22

• 10:48 arturo: rebase & resolve ugly git merge conflict in labs/private.git

2020-12-18

• 10:52 arturo: live-hacking local puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/650470 (T267966)

2020-12-14

• 19:27 bstorm: create temporary instance toolsbeta-test-io-unthrottled T267966
• 19:25 bstorm: created temporary instance toolsbeta-io-test-local T267966

2020-12-11

• 23:31 bstorm: increasing the output throttle for toolsbeta-test-k8s-haproxy-* nodes in order to figure out what's up with the timeouts

2020-12-10

• 08:58 dcaro: starting a new etcd instance completely from ansible playbook (etcd-8) (T267412)

2020-12-09

• 15:30 dcaro: Playing aronud adding a new etcd node (k8s-etcd-7) (T267412)

2020-12-04

• 11:17 dcaro: Created a new 'standardized' security froup for k8s from ansible toolsbeta-k8s-full-connectivity (T267412)
• 10:12 dcaro: Trying to create a whole new etcd member from ansible (T267412)

2020-11-23

• 14:17 dcaro: All control nodes re-imaged (T267140)
• 14:08 dcaro: Taking control-3 node out as control-6 is up and running (T267140)
• 11:12 dcaro: Launching control-6, to replace control-3 (T267140)
• 10:45 dcaro: Taking out control-2 node, replaced by control-5 (I saw one 503 reply on the proxy when creating control-5, fyi) (T267140)
• 10:32 dcaro: Creating new control-5 node (will replace control-2) (T267140)
• 09:58 dcaro: Remove control-1 node from the pool (was replaced by control-4) (T267140)
• 09:57 dcaro: Remove control-1 node from the pool (was replaced by control-4) (T267195)

2020-11-18

• 11:46 dcaro_: Modifying the security groupts to mirror tools (T267140)
• 10:50 dcaro_: Adding new control-4 node to the control cluster (T267140)

2020-11-17

• 15:32 dcaro: Creating new toolsbeta-test-k8s-control-4 node and adding it to the cluster (T267140)
• 12:09 Lucas_WMDE: <dcaro> 11:59:36 UTC – toolbeta up and running again, documented on the live doc for now, apsrever had the wrong config (T267140)
• 10:40 arturo: hand-edited /etc/kubernetes/manifests/kube-apiserver.yaml in all 3 k8s control nodes to account for new etcd servers (T267140)
• 08:58 dcaro: etcd hosts reimaged (T267140)
• 08:54 dcaro: etcd-4,5 and 6 are up and running, removing 1,2 and 3 (T267140)

2020-11-16

• 11:44 dcaro: etcd5 member added, creating instance toolsbeta-test-k8s-etcd6 and adding to the etcd cluster (T267140)
• 11:27 dcaro: Creating instance toolsbeta-test-k8s-etcd5 and adding to the etcd cluster (T267140)

2020-11-10

• 19:42 bstorm: safelisted "argocd" namespace with namespaceSelector for registry-admission controller
• 18:49 legoktm: associated floating IP to toolsbeta-docker-registry-01 and pointed DNS docker-registry.toolsbeta.wmflabs.org. at it
• 18:27 legoktm: creating toolsbeta-docker-imagebuilder-01 (T267616)
• 17:18 dcaro: launching instance toolsbeta-test-k8s-etcd-4 (T267140)
• 17:15 dcaro: removing unused toolsbeta-k8s-etcd prefix (we use toolsbeta-test-k8s-etcd) (T267140)
• 14:44 dcaro: taking down one of the test-k8s etcd nodes to reimage (T267140)

2020-11-06

• 23:44 bstorm: toolsbeta k8s cluster fully upgraded to 1.17.13 T263284
• 21:23 bstorm: upgrading toolsbeta-test-k8s-control-1 to k8s 1.17.13 T263284
• 15:56 dcaro: Deleting instances proxy-1 and proxy-2, that will finish the proxy rebuild (T267140)
• 15:53 dcaro: Removing proxy-1 and proxy-3 from hiera, proxy-3 stays as active and proxy-4 as backup (T267140)
• 13:18 dcaro: bringin up a new proxy-4 instance as slave (T267140)
• 13:18 dcaro: bringin up a new proxy-4 instance as slave

2020-11-05

• 16:40 dcaro: Moving active proxy from proxy-1 to proxy-3 (T267140)
• 15:54 dcaro: Adding toolsbeta-proxy-3 to the list of slave proxies in hiera (T267140)

2020-11-04

• 15:42 dcaro: re-creating the toolsbeta-proxy-03, used wrong image on the first try (T267140)
• 15:21 dcaro: creating new proxy instance toolsbeta-proxy-03
• 15:18 arturo: dropping project hiera config for `toollabs::checker_hosts`, `toollabs::proxy::ssl_certificate_name`, `toollabs::proxy::ssl_install_certificate` and `toollabs::proxy::web_domain`, no longer in use
• 15:16 arturo: dropping project hiera config for `toollabs::proxy::proxies`, no longer in use
• 11:46 dcaro: The k8s scheduler-01 fails to connect to etcd (not sure ever did), trying to fix

2020-11-03

• 16:04 arturo: add dcaro to the toolsbeta.admin LDAP group (T266068)
• 15:30 dcaro: T267121: Puppetmaster replaced, also removed old puppetdb master from hiera, testing
• 15:07 dcaro: Replacing old puppetmaster 02 and 03 from hiera with 04
• 10:55 dcaro: dcaro investigating puppet errors on toolsbeta-puppetdb-02

2020-11-02

• 13:35 arturo: added dcaro as projectadmin & user (T266068)

2020-10-29

• 22:20 legoktm: switched test tool over to use buildpack image (T265681)

2020-10-28

• 18:58 andrewbogott: deleting toolsbeta-puppetmaster-03 — seems broken and unused

2020-10-22

• 16:22 bstorm: created buildpack psp for T265557

2020-09-10

• 09:17 arturo: force-rebooting toolsbeta-test-haproxy-2 (unresponsive)
• 09:15 arturo: livehacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/626133 (T250172)
• 09:00 arturo: tainted/labeld toolsbeta-test-k8s-ingress-1 (and -2) in the k8s cluster (T250172)
• 08:59 arturo: added toolsbeta-test-k8s-ingress-1 (and -2) to the k8s cluster (T250172)

2020-09-09

• 11:50 arturo: after force-rebooting everything, the k8s cluster seems to have recovered itself. magic.
• 11:45 arturo: force-rebooting the 3 k8s etcd nodes. They seem down
• 11:42 arturo: actually, the whole k8s cluster seems down? the API seems down at least
• 11:39 arturo: all 3 k8s control nodes seem in bad shape. Wont let me ssh in, or use the console access. Try force-rebooting them
• 11:27 arturo: created 2 VMs: toolsbeta-test-k8s-ingress-1 and toolsbeta-test-k8s-ingress-2 (T250172)
• 11:25 arturo: created new server group toolsbeta-k8s-ingress (T250172)
• 11:24 arturo: created new puppet prefix `toolsbeta-test-k8s-ingress` (T250172)

2020-07-15

• 21:35 bstorm: set all of toolsbeta to mount NFS 4.2 except the bastion T257945

2020-07-14

• 22:28 bstorm: rebooting toolsbeta-sgebastion-04 during NFS testing thing

2020-07-08

• 11:08 arturo: live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/610029 (T234617)

2020-06-26

• 12:12 arturo: puppetmaster live-hacking with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/608005 (T120210)

2020-06-24

• 12:55 arturo: live-hacking puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/607279 (T120225)
• 12:23 arturo: live-hacking puppetmaster with exim prometheus stuff (T175964)
• 11:31 arturo: live-hack the puppetmaster with https://gerrit.wikimedia.org/r/c/operations/puppet/+/607320 (T175964)
• 11:26 arturo: add TXT record `"v=spf1 mx -all"` T120225
• 11:24 arturo: fix MX record for toolsbeta.wmflabs.org (missing trailing dot) T120225

2020-06-23

• 13:10 arturo: added herron to the test tool for email testing
• 11:36 arturo: removing `benapetr` and adding myself to the test tool
• 11:02 arturo: setting `profile::toolforge::mail_domain: toolsbeta.wmflabs.org` in toolsbeta-mail puppet prefix
• 10:55 arturo: allow ingress smtp/smtps traffic in the MTA security group
• 10:52 arturo: created MX record pointing to mail.toolsbeta.wmflabs.org
• 09:43 arturo: restarted nginx in toolsbeta-acme-chief-01 to pickup new certificate, otherwise clients won't accept its TLS cert
• 09:38 arturo: live-hacking toolsbeta-puppetmaster-04 with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/607251

2020-06-16

• 22:54 bd808: Building webservice 0.72

2020-06-15

• 21:54 bstorm_: removed killgridjobs.sh from toolsbeta bastion T157792
• 17:52 bd808: Building webservice 0.71

2020-06-12

• 19:41 bstorm_: set `profile::wmcs::nfsclient::mode: soft` on toolsbeta-workflow-test T127559

2020-06-11

• 12:42 arturo: introduce puppet profile 'toolsbeta-docker-registry' and relocate some hiera config there
• 12:39 arturo: for the record, k8s etcd servers certificate changed (puppet based) and k8s just kept working
• 12:35 arturo: according to `aborrero@cloud-cumin-01:~$ sudo cumin --force -x 'O{project:toolsbeta}' 'run-puppet-agent'` we are mostly back in business
• 12:14 arturo: try switching all VMs to toolsbeta-puppetmaster-04
• 12:14 arturo: poweroff toolsbeta-puppetmaster-03
• 12:12 arturo: copy over labs/private from toolsbeta-puppetmaster-03 to toolsbeta-puppetmaster-04
• 11:53 arturo: create VM toolsbeta-puppetmaster-04
• 11:35 arturo: try reinstalling the python3 stack in toolsbeta-puppetmaster-03, because everything python-related segfaults
• 11:33 arturo: reboot toolsbeta-puppetmaster-03 to try cleaning up potential kernel/filesystem problems
• 11:32 arturo: apparently every python script segfaults in toolsbeta-puppetmaster-03
• 11:27 arturo: puppetdb wasn't the problem. The problem is puppet-enc segfaulting in toolsbeta-puppetmaster-03
• 11:21 arturo: puppet not working bc puppetdb, run `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb`

2020-06-04

• 21:06 andrewbogott: added krenair to toolsbeta.admin group in ldap

2020-05-28

• 11:27 arturo: cleanup livehackings
• 10:31 arturo: livehacking puppetmaster and toolsbeta-proxy-1 to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/599139 (T253816)
• 10:30 arturo: livehacking puppetmaster to test https://gerrit.wikimedia.org/r/c/operations/puppet/+/599139

2020-05-27

• 12:02 arturo: the k8s cluster is now running v1.16.10 (T246122)
• 11:05 arturo: trying `modules/kubeadm/files/wmcs-k8s-node-upgrade.py --control toolsbeta-test-k8s-control-1 --project toolsbeta --domain eqiad.wmflabs --src-version 1.15 --dst-version 1.16.10 -n toolsbeta-test-k8s-worker-1 -n toolsbeta-test-k8s-worker-2 -n toolsbeta-test-k8s-worker-3` (T246122)
• 11:02 arturo: upgraded the rest of the k8s control plane nodes to 1.16.10 (T246122)
• 10:58 arturo: running `aborrero@toolsbeta-test-k8s-control-1:~ $ sudo apt-get install kubelet -y` in the 1.16 version from the component repo (T246122)
• 10:58 arturo: running `aborrero@toolsbeta-test-k8s-control-1:~ $ sudo -i kubeadm upgrade apply v1.16.10` and this time it works! (T246122)

2020-05-26

• 16:17 bstorm_: fix incorrect volume name in kubeadm-config T246122
• 15:02 arturo: first k8s upgrade failed for yet-to-be-known reasons (T246122)
• 14:54 arturo: `aborrero@toolsbeta-test-k8s-control-1:~ $ sudo -i kubeadm upgrade apply v1.16.10` (T246122)
• 14:54 arturo: bump installed version of kubeadm and kubectl to 1.16.10 (T246122)
• 09:57 arturo: installing kubectl/kubeadm 1.16.9 on k8s worker nodes (T246122)
• 09:56 arturo: installing kubectl/kubeadm 1.16.9 on k8s control nodes (T246122)
• 09:30 arturo: set `profile::wmcs::kubeadm::component: 'thirdparty/kubeadm-k8s-1-16'` at project level for trying T246122
• 09:25 arturo: `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb` broken puppet in this project because puppetdb is down again

2020-05-21

• 22:14 bd808: Building tools-webservice 0.70 via wmcs-package-build.py

2020-05-19

• 12:20 arturo: trying to install tesseract 4.1.0 in toolsbeta-sgebastion-04 (T247422)
• 10:18 arturo: `aborrero@toolsbeta-puppetdb-02:~$ sudo systemctl restart puppetdb`

2020-05-15

• 20:48 bstorm_: found an error in the new version of maintain-kubeusers, removing the deployment for now T246059
• 20:35 bstorm_: updating the maintain-kubeusers image to be able to control admin accounts

2020-05-14

• 12:09 arturo: created puppet prefix toolsbeta-acme-chief in horizon (T252762)
• 12:08 arturo: created toolsbeta-acme-chief-01 VM (T252762)

2020-05-12

• 18:35 bstorm_: upgraded to using typha and rolled back to not doing so -- no affect on existing network T250863
• 17:44 bstorm_: set the calico version to v3.14.0 because the new liveness probe isn't compatible with the old version. T250863
• 17:36 bstorm_: deployed an updated bit of yaml for calico without upgrading the version first T250863

2020-05-08

• 12:48 arturo: allocated floating IP `185.15.56.12` for the VM `toolsbeta-email-01` and FQDN `mail.toolsbeta.wmflabs.org` (T120225)
• 12:24 arturo: added puppet prefix `toolsbeta-email` (T120225)

2020-05-07

• 16:33 arturo: livehack toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/594945 (T251297 and T250866)
• 12:36 arturo: cleanup livehacks in toolsbeta-puppetmaster-03
• 11:12 arturo: livehack toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/594925 and https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/594926 (T251297 and T250866)

2020-05-06

• 19:11 bstorm_: updated toollabs-webservice to 0.69 for toolsbeta
• 09:58 arturo: livehacking toolsbeta-puppetmaster-03 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/594471 (T251297)

2020-05-05

• 10:04 arturo: add herron as user and projectadmin, we will work on the email setup (T120225)
• 09:59 arturo: created VM toolsbeta-mail-01 (T120225)

2020-05-04

• 13:02 arturo: `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb.service` trying to bring back puppetdb, which is preventing puppet agent runs in the whole project

2020-04-29

• 19:48 bstorm_: ran the scary rewrite-psp-preset.sh script across toolsbeta T247455

2020-04-20

• 14:47 arturo: added joakino to toolsbeta.admin LDAP group
• 12:06 arturo: installing tools-webservice v0.68 for testing
• 11:05 arturo: poweroff `toolsbeta-services-01`. I suspect this VM is not in use because no puppet role is in used there
• 10:58 arturo: run `aborrero@toolsbeta-puppetdb-02:~ $ sudo systemctl restart puppetdb` the service was in failed state, causing puppet failures across the whole project

2020-04-10

• 19:32 bstorm_: deployed webservice 0.67 T249843
• 18:59 bstorm_: delete toolsbeta-gitlab-01 and build toolsbeta-workflow-test T249946
• 00:40 bd808: REbooting toolsbeta-sgebastion-04. NFS seemed messed up

2020-04-08

• 01:10 bstorm_: upgrade toollabs-webservice to 0.66 for qa T249390

2020-03-31

• 23:39 bstorm_: deployed toollabs-webservice-0.65 to toolsbeta

2020-03-30

• 10:35 arturo: remove local changes in the puppet tree in toolsbeta-puppetmaster-03 (docker mount point)
• 10:30 arturo: remove puppet prefixes `toolsbeta-test-proxy`, `toolsbeta-k8s-master`, `toolsbeta-flannel-etcd`, no longer in use

2020-03-24

• 18:45 jeh: cleanup and remove toolsbeta-elastic7-[1,2,3] VMs (re-configuring hypervisor for local storage) T243327

2020-03-19

• 23:18 Krenair: Shut down toolsbeta-puppet(db-01|master-02) - T241719
• 19:20 arturo: live-hacking toolsbeta-proxy-1 with https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/579952 (T234617)

2020-03-16

• 21:38 bstorm_: removed lots of hiera related to the legacy k8s cluster T246689
• 19:45 bstorm_: deleting toolsbeta-worker-1001, toolsbeta-k8s-master, toolsbeta-flannel-etcd-01 and toolsbeta-k8s-etcd-01 T246689
• 19:07 bstorm_: shutting down toolsbeta-flannel-etcd-01 T246689
• 19:06 bstorm_: shutting down toolsbeta-worker-1001, toolsbeta-k8s-master and toolsbeta-k8s-etcd T246689
• 14:37 arturo: live-hacking the toollabs-webservice package in toolsbeta-sgewebgrid-lighttpd-0901 as well
• 14:22 arturo: live-hacking the toollabs-webservice package in toolsbeta*-sgebastion-04 with https://gerrit.wikimedia.org/r/c/operations/software/tools-webservice/+/578413 (T234617)
• 14:22 arturo: live-hacking the toollabs-webservice package in tools-sgebastion-04 with https://gerrit.wikimedia.org/r/c/operations/software/tools-webservice/+/578413 (T234617)
• 13:49 arturo: deleting 50 jobs of the `test` tool in the grid to leave room for other tests
• 13:18 arturo: live-hack toolsbeta-puppetmaster-02 with https://gerrit.wikimedia.org/r/c/operations/puppet/+/578406 (T234617)

2020-03-11

• 21:32 bstorm_: deployed jobutils_1.39 and miscutils_1.39 to toolsbeta

2020-03-09

• 13:11 arturo: created VM `toolsbeta-legacy-redirector` (T247236)
• 13:08 arturo: instance quota was full, bump it from 35 to 40

2020-03-06

• 16:22 bstorm_: updating maintain-kubeusers image to filter invalid tool names

2020-03-05

• 21:22 bstorm_: updated maintain-kubeusers to the latest version for toolsbeta only to live test

2020-02-27

• 19:19 bstorm_: upgraded toollabs-webservice to 0.64 on stretch-toolsbeta for testing
• 16:03 jeh: create 3 new VMs toolsbeta-elastic7-0[1,2,3]
• 16:00 jeh: increase CloudVPS quota instance count for new elasticsearch servers

2020-02-26

• 20:35 bstorm_: hard rebooting the grid master for toolsbeta
• 20:20 jeh: restart toolsbeta-sgegrid-shadow

2020-02-18

• 23:20 bstorm_: added toolsbeta-sgegrid-master.toolsbeta.eqiad1.wikimedia.cloud and toolsbeta-sgegrid-shadow.toolsbeta.eqiad1.wikimedia.cloud to gridengine admin host lists

2020-02-10

• 21:19 bstorm_: upgraded toollabs-webservice package for stretch toolsbeta to 0.62 T244293 T244289 T234617 T156626

2020-02-07

• 23:07 bstorm_: upgraded toollabs-webservice for stetch toolsbeta to 0.60 T244611
• 21:09 bstorm_: upgraded toollabs-webservice package for stretch toolsbeta to 0.59 T244293 T244289 T234617 T156626

2020-01-23

• 03:14 bd808: Demoted projectadmins not listed in the "roots" sudoer policy to project members just to avoid random confusion
• 03:06 bd808: Added legoktm to "roots" sudoer policy
• 02:53 bd808: Added legoktm as project admin

2020-01-22

• 11:59 arturo: remove toolviews scripts from toolsbeta-proxy-{1,2}, source of cronspam

2020-01-21

• 12:49 arturo: cleanup livehackings in toolsbeta-sgebastion-04 and toolsbeta-proxy-1
• 09:40 arturo: livehacking toolsbeta-sgebastion-04 (https://gerrit.wikimedia.org/r/c/566045 and https://gerrit.wikimedia.org/r/c/565575) and toolsbeta-proxy-1 (https://gerrit.wikimedia.org/r/c/565556) for testing T234617

2020-01-17

• 12:52 arturo: livehack toolsbeta-puppetmaster-02 to test https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/565556 (T234617)
• 10:37 arturo: enabling puppet agent in toolsbeta-proxy-1 which was disabled without reason since 2019-12-02 (probably by me)

2020-01-16

• 23:13 bstorm_: updated toollabs-webservice to 0.58 for stretch to test things out
• 12:07 arturo: live-hack tools-webservice in tools-sgebastion-04 to test https://gerrit.wikimedia.org/r/c/565259 (T242719)

2020-01-14

• 02:15 andrewbogott: rebooting toolsbeta-sgecron-01 and toolsbeta-test-k8s-etcd-3 to get nfs unstuch

2020-01-13

• 16:41 bstorm_: There was a filesystem unclean and other problems on the "old cluster" worker node 1001. Rebooting it in case that helps.

2020-01-10

• 21:05 bstorm_: updated toollabs-webservice package to 0.55 for testing

2020-01-07

• 15:51 bstorm_: changed kubeadm-config to use a list instead of a hash for extravols on the apiserver in the new k8s cluster T242067

2020-01-06

• 21:42 bstorm_: disabled rpcbind on toolsbeta-sgebastion-04 to test some things

2020-01-03

• 17:46 bstorm_: stashed uncommitted changes on the puppetmaster because they seem to be things that are already merged
• 11:27 arturo: [new k8s] cadvisor is running in the metrics namespace now (T237643)

2020-01-02

• 22:37 bstorm_: Deleting the massive number of test ingresses for tool-fourohfour so the ingress controllers aren't moving so slowly.
• 22:19 bstorm_: Changed the ingress-admission ValidatingWebhookConfiguration to check extensions as well as networking API groups

2019-12-17

• 00:14 bstorm_: Fully enabled encryption at rest for toolsbeta kubernetes

2019-12-16

• 23:03 bstorm_: updated the kubeadm-config configmap to match the new init file

2019-12-04

• 13:02 arturo: drop puppet prefix `toolsbeta-grid-master`, deprecated and no longer in use
• 12:50 arturo: drop puppet prefix `toolsbeta-bastion`, deprecated and no longer in use

2019-12-02

• 10:38 arturo: create wildcard DNS record for `*.toolsbeta.wmflabs.org` for use by the new k8s cluster
• 10:34 arturo: manually scale nginx-ingress deployment to 5 replicas (T239405)

2019-11-25

• 10:30 arturo: add puppet cert SANs via hiera to toolsbeta-test-k8s-etcd nodes (T238655)

2019-11-21

• 14:15 arturo: upgrade new k8s cluster to 1.15.6 using kubeadm (plus kubelet)

2019-11-15

• 14:46 arturo: stop live-hacks on toolsbeta-test-k8s-haproxy-1 T237643

2019-11-14

• 10:32 arturo: live-hacking toolsbeta-test-k8s-haproxy-1 to point to just the k8s apiserver in control-1 Turn on --v=10 in control-1 for extended debug

2019-11-08

• 19:36 bstorm_: rebooted the proxy server just in case that fixes something.
• 11:58 arturo: adding `profile::toolforge::bastion::nproc: 100` to puppet prefix `toolsbeta-sgebastion` (T236202)
• 11:38 arturo: new k8s: refresh deployment for nginx-ingress with latest changes from puppet

2019-11-07

• 21:55 bstorm_: killed pods for ingress admission controller to upgrade to new image T215531

2019-11-06

• 22:39 bstorm_: upgraded repo version of toollabs-webservice in toolsbeta-stretch to 0.49 -- changes for the new k8s cluster T215531
• 19:09 bstorm_: added profile::toolforge::proxies in global hiera to try and figure out why it won't let anything use redis T237443
• 18:53 bstorm_: launching toolsbeta-proxy-2 on a hunch that the config doesn't work well as a standalone T237443
• 18:46 bstorm_: rebooting toolsbeta-proxy-1 trying to convince redis it is not a read replica T237443
• 18:29 bstorm_: stopped broken kube-proxy service on toolsbeta-proxy-1 (should probably be puppetized)
• 17:35 bstorm_: changing some hiera to work with new proxy host
• 12:44 arturo: created VM toolsbeta-proxy-1 (T237443)

2019-11-05

• 22:50 bstorm_: deployed the new maintain-kubeusers to toolsbeta T215531 T228499

2019-10-25

• 23:41 bstorm_: Deployed custom webhook controllers for registry and ingress checking to toolsbeta-test kubernetes cluster T215531 T215678 T234231
• 16:15 bstorm_: rebooting toolsbeta-test-k8s-worker-1 and -2

2019-10-23

• 12:04 arturo: created 2 new VMs `toolsbeta-test-k8s-worker-[1,2]` T236074
• 11:56 arturo: point FQDN `k8s.toolsbeta.eqiad1.wikimedia.cloud` to `toolsbeta-test-k8s-haproxy-1` (T236074)
• 11:20 arturo: re-create VM `toolsbeta-test-k8s-haproxy-1` to use new puppet profile (T236074)
• 11:10 arturo: re-create VM `toolsbeta-test-k8s-haproxy-2` to test https://gerrit.wikimedia.org/r/545532 (T236074)

2019-10-22

• 17:43 arturo: re-create VM `toolsbeta-test-k8s-control-1` T236074
• 15:48 arturo: point DNS record `k8s.toolsbeta.eqiad1.wikimedia.cloud` to the first controller node for the bootstrap T236074
• 15:30 arturo: created puppet prefix `toolsbeta-test-k8s-control` and delete `toolsbeta-test-k8s-master` T236074
• 12:27 arturo: refreshed puppet prefix `toolsbeta-test-k8s-control` with latest info T236074
• {{safesubst:SAL entry|1=12:26 arturo: created 3 VMs `toolsbeta-test-k8s-control-{1,2,3}` T236074}}
• 12:15 arturo: refresh IP addr of FQDN `k8s.toolsbeta.eqiad1.wikimedia.cloud` T236074
• 12:14 arturo: delete FQDN `toolsbeta-k8s-master.toolsbeta.wmflabs.org` T236074
• {{safesubst:SAL entry|1=11:57 arturo: created 2 new VMS `toolsbeta-test-k8s-haproxy-{1,2}` T236074}}
• 11:54 arturo: created puppet prefix `toolsbeta-test-k8s-haproxy` and delete `toolsbeta-test-k8s-lb` T236074

2019-10-21

• 15:13 arturo: refresh config in prefix puppet `toolsbeta-test-k8s-etcd` to account for new servers T236074
• {{safesubst:SAL entry|1=15:07 arturo: create 3 VMs toolsbeta-test-k8s-etcd-{1,2,3} T236074}}
• 14:58 arturo: deleting all toolsbeta-test-* VMs (master, worker, etcd, lb) T236074

2019-10-18

• 16:33 arturo: created DNS zone `toolsbeta.eqiad1.wikimedia.cloud`
• 09:06 arturo: remove puppet prefix toolsbeta-valhallasw-puppet-compiler (unused)
• {{safesubst:SAL entry|1=09:00 arturo: remove puppet prefix toolsbeta-arturo-k8s-{etcd,master,worker} (unused)}}
• {{safesubst:SAL entry|1=08:59 arturo: refresh role for servers in toolsbeta-test-k8s-{master,worker}}}
• 08:58 arturo: remove puppet prefix etcd-k8s-ctest (unused)

2019-10-14

• 12:26 arturo: delete VM `toolsbeta-test-proxy-01` no longer required
• 12:26 arturo: created security group arturo-test-dynamicproxy-backend to tests stuff related to T234037

2019-10-09

• 11:59 arturo: re-create toolsbeta-test-proxy-01 as Debian Buster (T235059)

2019-10-08

• 14:14 arturo: created puppet prefix `toolsbeta-test-proxy` for testing stuff related to T234037
• 12:27 arturo: created VM toolsbeta-test-proxy-01 for testing stuff related to T234037

2019-10-07

• 19:12 Krenair: reboot toolsbeta-sgecron-01 toolsbeta-sgewebgrid-generic-0901 toolsbeta-sgewebgrid-lighttpd-0901 due to nfs stale issue

2019-09-25

• 23:31 bd808: Updated user list for "roots" sudoer policy
• 23:30 bd808: Granted Krenair projectadmin

2019-09-05

• {{safesubst:SAL entry|1=15:08 zhuyifei1999_: `sudo truncate -s 0 /var/log/exim4/paniclog` on toolsbeta-{sgewebgrid-{lighttpd,generic}-0901,sgecron-01}.toolsbeta.eqiad.wmflabs because of email spam}}

2019-08-12

• 20:40 phamhi: toolsbeta-test-puppet-sandbox instance created for T230147

2019-08-09

• 10:51 arturo: rebalance load: reallocating toolsbeta-sgewebgrid-lighttpd-0901 from cloudvirt1018 to cloudvirt1003

2019-07-24

• 20:48 bstorm_: rebuilt toolsbeta-test cluster with the internal version of the pause container T228887 T215531
• 19:02 bstorm_: doing a clean rebuild of the toolsbeta-test-k8s cluster

2019-07-18

• 16:04 arturo: re-create VMs toolsbeta-test-k8s-{master,worker}-*
• 12:47 arturo: create toolsbeta-test-k8s-etcd-2 as buster to check status of latest puppet code (T226098)
• 12:00 arturo: create toolsbeta-test-k8s-worker-2 as buster to check status of latest puppet code
• {{safesubst:SAL entry|1=09:28 arturo: re-create toolsbeta-test-k8s-master-{1,2,3} as buster to test T228267}}

2019-07-17

• 09:51 arturo: re-create VM toolsbeta-test-k8s-worker-1 as Debian Buster T215531
• 09:13 arturo: create VM toolsbeta-test-k8s-master-4 (Debian Buster) T215531

2019-07-15

• 12:29 arturo: create `toolsbeta-test-k8s-etcd` puppet prefix
• 12:27 arturo: create `toolsbeta-test-k8s-etcd-1` VM T215531

2019-07-03

• 10:49 arturo: recreate `toolsbeta-test-k8s-master-1` VM (T215531)
• 09:32 arturo: create `toolsbeta-test-k8s-worker-1` VM and a puppet prefix for it (T215531)
• 09:22 arturo: delete all `toolsbeta-arturo-k8s-*` instances. We no longer require them per new approach at T215531

2019-07-02

• 17:24 arturo: `aborrero@toolsbeta-test-k8s-lb-01:~ $ sudo generate_haproxy_default.sh` (T215531)
• 10:32 arturo: re-creating toolsbeta-test-k8s-master-1 (T215531) for it to be created without swap

2019-07-01

• 17:13 arturo: re-creating instance `toolsbeta-test-k8s-master-1` with more CPU for T215531
• 17:03 arturo: updated FQDN `toolsbeta-k8s-master.toolsbeta.wmflabs.org` with 172.16.6.9 (the new LB VM) for T215531
• 17:02 arturo: re-creating instance `toolsbeta-test-k8s-lb-01` with more CPU for T215531
• 16:58 arturo: add puppet prefix `toolsbeta-test-k8s-lb` for T215531
• 11:50 arturo: add sssd hiera config for `toolsbeta-test-k8s-master` prefix

2019-06-28

• 19:10 bstorm_: T215531 removed toolsbeta-arturo-k8s-master-2/3 and added toolsbeta-test-k8s-master-1 for testing kubeadm

2019-06-25

• 10:35 arturo: create puppet prefix `toolsbeta-arturo-k8s-worker` for T215531
• 10:35 arturo: create 2 VMs toolsbeta-arturo-k8s-worker-[1,2] for T215531

2019-06-21

• 11:42 arturo: re-create 3 VMs toolsbeta-arturo-k8s-etcd-[1-3] to test latest puppet code in T226098

2019-06-19

• 10:39 arturo: add myself to the `toolsbeta.admin` LDAP group (T225303)

2019-06-14

• 16:24 bstorm_: Manually failed "back" to the toolsbeta-sgegrid-master to get the grid functioning again in toolsbeta
• 16:03 bstorm_: T221721 hard rebooted toolsbeta-sgegrid-master because it had oomkilled basically everything
• 15:55 bstorm_: T221721 deleted toolsbeta-proxy-01 until it can be actively worked on.
• 15:51 bstorm_: deleted toolsbeta-k8s-lb-01 since it isn't being actively worked on just now

2019-06-06

• 12:14 arturo: T215531 create 3 VMs `toolsbeta-arturo-k8s-etcd-[1-3]`
• 12:13 arturo: T215531 add `toolsbeta-arturo-k8s-etcd`* puppet prefix
• 12:12 arturo: T215531 add `toolsbeta-arturo-k8s-test` puppet prefix

2019-06-05

• 12:40 arturo: rebase git repos in toolsbeta-puppetmaster-02. There was some rebase problems in labs/private that required me re-creating by hand one of the [local] patches (puppetdb secrets)
• 12:33 arturo: drop VM instances toolsbeta-k8s-master-arturo-[1-3] and create toolsbeta-arturo-k8s-master-[1-3] T215531
• 12:32 arturo: drop puppet prefix `toolsbeta-k8s-master-arturo` and create `toolsbeta-arturo-k8s-master` since there is also `toolsbeta-k8s-master` which get applied to my VMs T215531
• 11:42 arturo: create VM `toolsbeta-k8s-master-arturo-3` for T215531 (so I have 3 master nodes in this k8s deployment)
• 11:38 arturo: delete instances arturo-sgeexec-sssd-test-2, arturo-sgeexec-sssd-test-1, arturo-bastion-sssd-test, unused

2019-05-24

• 11:49 arturo: T224273 create `toolsbeta-k8s-master-arturo` puppet prefix in horizon
• 11:45 arturo: T224273 create toolsbeta-k8s-master-arturo-[12] stretch VMs
• 11:17 arturo: install by hand some openstack client packages that puppet would refuse to install in toolsbeta-k8s-master-01
• 11:12 arturo: mangle sources.list to handle some apt warnings related to missing repos, etc in toolsbeta-k8s-master-01:
• 11:12 arturo: mangle sources.list to handle some apt warnings related to missing repos, etc

2019-05-07

• 10:22 arturo: T219362 drop the `toolsbeta-exec` puppet prefix
• 10:20 arturo: T219362 drop the `toolsbeta-webgrid-generic` puppet prefix
• 10:19 arturo: T219362 drop the `toolsbeta-webgrid-lighttpd` puppet prefix

2019-04-25

• 04:17 andrewbogott: edited resolv.conf on unpuppetized instances to use the new nameserver: toolsbeta-docker-registry-01, toolsbeta-k8s-lb-01, toolsbeta-proxy-01, toolsbeta-puppetdb-01, toolsbeta-sgegrid-master

2019-04-12

• 23:34 mutante: - toolsbeta-k8s-master-01 - was out of disk space on / , puppet failed to run because out of disk, rename existing syslog.1.gz, gzip syslog.1, rename existing daemon.log.1.gz, gzip daemong.log.1
• 00:05 andrewbogott: migrating remaining VMs to eqiad1-r

2019-03-25

• 18:00 bd808: All Trusty instances shutdown and now in process of deleting
• 17:42 bd808: Preparing to shutdown beta Trusty job grid

2019-03-22

• 13:59 arturo: create VMs arturo-sgeexec-sssd-test-[12] for testing T218126

2019-03-15

• 10:23 arturo: create VM `arturo-bastion-sssd-test` (T218126)

2019-02-20

• 14:58 andrewbogott: moving toolsbeta-grid-master and toolsbeta-puppetmaster-02 to labvirt1003

2019-02-14

• 18:30 andrewbogott: moving toolsbeta-puppetdb-01 to labvirt1002

2018-12-04

• 18:43 arturo: some hiera keys reallocated, see https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/477607/

2018-11-26

• 13:26 arturo: T210098 VM=toolsbeta-sgebastion-03
• 13:25 arturo: T210098 install systemd239 from stretch-backports and restart VM

2018-11-08

• 10:01 arturo: make myself projectadmin to test toolforge stuff on stretch (specifically T207970)

2018-10-22

• 21:20 bstorm_: launched a stretch/sonofgridengine master server

2018-09-19

• 20:11 bstorm_: toolsbeta-puppetmaster-02 is now the puppetmaster and puppetdb works for toolsbeta -- T200557
• 17:24 bstorm_: new puppetmaster is toolsbeta-puppetmaster-02, however, manual changes are required on each client, so it will be broken for a bit (enabling puppetdb for T200557)
• 17:06 bstorm_: working on replacing puppetmaster with one running stretch, as part of adding puppetdb

2018-07-22

• 14:28 zhuyifei1999_: backed up Neha16's changes to toolsbeta-bastion-01:/usr/lib/python2.7/dist-packages/toollabs to toollabs.bak in the same dir via cp -a, and re-install webservice code on the bastion to debug T156626

2018-07-18

• 10:46 harej: Deleted toolsbeta-flynn-01

2018-07-12

• 23:06 bstorm_: Got the grid master running

2018-06-28

• 16:34 chicocvenancio: adding harej as root for flynn testing

2018-06-27

• 22:35 chicocvenancio: add harej as project admin to test Flynn stuff

2018-06-22

• 22:26 chicocvenancio: reconfigured toolsbeta-paws-master-01 kubelet to test image pruning
• 09:39 zhuyifei1999_: fixed that by running `sudo mv /var/lib/puppet/ssl /var/lib/puppet/ssl.bak` then following the red instructions
• 09:33 zhuyifei1999_: puppet is broken on toolsbeta-bastion-01, investigating
• 09:03 zhuyifei1999_: killing and rebuilding toolsbeta-bastion-01
• 08:31 zhuyifei1999_: on toolsbeta-bastion-01, killed /etc/apt/sources.list.d/jonathonf-python-2_7-trusty.list ppa, downgraded python from 2.7.14 to 2.7.5, and reinstalled toollabs-webservice
• 07:56 andrewbogott: someone removed /usr/bin/webservice

2018-05-15

• 07:26 zhuyifei1999_: applied 5324236 via toolsbeta-puppetmaster-01 T190893
• 05:28 zhuyifei1999_: Making project puppetmaster at toolsbeta-puppetmaster-01

2018-05-08

• 02:18 zhuyifei1999_: manually created flannel etcd key T190893

2018-05-07

• 19:01 zhuyifei1999_: install kubernetes-client on toolsbeta-worker-1001 to debug stuffs
• 18:41 zhuyifei1999_: rebuilding toolsbeta-k8s-etcd-01
• 17:58 zhuyifei1999_: cleanup from maintain-kubeusers using the wrong project to create tool home dirs: `find /data/project/ -mindepth 1 -maxdepth 1 -type d \! -user 0 | (while read dir; do id toolsbeta.`basename $dir` 2> /dev/null || sudo rm -rfv $dir; done)`
• 16:41 zhuyifei1999_: rebuild toolsbeta-k8s-master-01 because I can't figure out why puppet can't update maintain-kubeusers.systemd

2018-05-06

• 04:06 zhuyifei1999_: locally patched `/usr/lib/python2.7/dist-packages/toollabs/common/tool.py` on bastion and webgrid-lighttpd

2018-05-05

• 19:51 zhuyifei1999_: `systemctl mask maintain-kubeusers` because it's causing a mess, tries to get the tool list from toolforge T190893
• 18:40 zhuyifei1999_: to unblock k8s testing while waiting on https://gerrit.wikimedia.org/r/430539, installed the package directly on `toolsbeta-k8s-master-01` with `$ sudo apt install python3-yaml`

2018-05-02

• 21:02 zhuyifei1999_: copy over labs/private:/hieradata/labs/tools/common.yaml to project puppet hiera
• 20:37 bd808: Added Neha16 as a project admin for work on T175768
• 20:31 zhuyifei1999_: nuke webservice instances and rebuild
• 20:31 zhuyifei1999_: Added k8s_infrastructure_users to project hiera on horizon T192618

2018-04-20

• 00:20 zhuyifei1999_: deleted all instances I just created except k8s master because of chicken-and-egg problem

2018-04-19

• 22:10 zhuyifei1999_: the trusty instances ask me for my password. the jessie instances don't like my ssh key. :(
• 21:59 zhuyifei1999_: got 'Error: RecordSet belongs in a child zone: toolsbeta.wmflabs.org', using tools-beta.wmflabs.org instead
• 21:57 zhuyifei1999_: Add proxy toolsbeta.wmflabs.org => toolsbeta-proxy-01.toolsbeta.eqiad.wmflabs
• 21:43 zhuyifei1999_: Start creating instances for webservice setup T190893

2018-03-30

• 22:40 zhuyifei1999_: copied over many prefix puppet configuration in horizon from toolforge T190893

2018-03-14

• 18:07 chicocvenancio: updated paws-beta k8s cluster and nodes to v1.9.4 for T189680

2018-03-05

• 19:33 chicocvenancio: added Zhuyifei1999 as project admin

2018-02-09

• 01:11 bd808: Removed Yuvipanda at user request (T186289)

2017-08-07

• 14:09 andrewbogott: deleted etcd-k8s-CTEST and k8s-master-CTEST

2017-04-26

• 15:38 madhuvishy: add Madhuvishy as projectadmin

2016-10-07

• 19:30 valhallasw`cloud: (puppet certs, to be precise)
• 19:30 valhallasw`cloud: fixed certs on toolsbeta-vagrant3-scfc.toolsbeta.eqiad.wmflabs

2016-10-04

• 19:31 valhallasw`cloud: puppet is broken due to incorrect certificates. Cleaning up ('puppet cert clean toolsbeta-webgrid-lighttpd-1406.toolsbeta.eqiad.wmflabs' on puppetmaster3, 'rm -f /var/lib/puppet/client/ssl/certs/toolsbeta-webgrid-lighttpd-1406.toolsbeta.eqiad.wmflabs.pem' on host, for all hosts that I got emails for)

2016-09-08

• 17:11 bd808: Added BryanDavis (self) to project as admin

2016-08-29

• 19:20 yuvipanda: reboot toolsbeta-master, seems, uh, stuck
• 19:18 yuvipanda: reboot toolsbeta-mail, seems, uh, stuck
• 18:48 yuvipanda: reboot toolsbeta-puppetmaster3, puppet run process became Zommmmbiiiieeee, ate all my brains

2016-07-03

• 15:02 yuvipanda: migrating toolsbeta-valhallasw-puppet-compiler to labvirt1011 to ease pressure on labvirt1010

2016-05-27

• 18:57 valhallasw`cloud: sudo qconf -Ae /var/lib/gridengine/etc/exechosts/toolsbeta-exec-1209.toolsbeta.eqiad.wmflabs

2016-05-26

• 15:08 valhallasw`cloud: toolsbeta-mail has high load (1.0) without clear origin, so rebooting the host

2015-10-13

• 19:21 valhallasw`cloud: started building toolsbeta-bastion.

2015-09-07

• 18:50 valhallasw`cloud: role::bastion is now applied on -exec-101. Now for the package_builder manifest...
• 18:30 valhallasw`cloud: applied role::toollabs::bastion on toolsbeta-exec-101 (spinning up a whole new instance will take ages)

July 4

• 12:57 valhallasw`cloud: restarting toolsbeta-webproxy, no response on port 22

July 2

• 14:55 valhallasw`cloud: toolsbeta-webproxy does not respond at all to SSH; rebooting

July 1

• 19:47 valhallasw`cloud: still can't login :/ not sure if this is a remainder of the NFS failure or something else; maybe a puppet run will solve it?
• 19:44 valhallasw`cloud: restarting toolsbeta-exec-01 and toolsbeta-mail as I can't login

June 7

• 14:44 valhallasw: updated /var/lib/git/operations/puppet to make sure the other hosts get the memo
• 14:42 YuviPanda: run sudo sed -i 's/GlobalSign_CA.pem/ca-certificates.crt/' /etc/ldap/ldap.conf on toolsbeta-puppetmaster3 to fix broken LDAP TLS config

May 11

• 18:14 valhallasw: building toolsbeta-pbuilder to experiment with pbuilder for building packages

May 2

• 11:11 valhallasw`cloud: commenting out include ::elasticsearch::ganglia in role::logstash seems to work. I think we have to write our own tools logstash roles anyway in the end, as the role::logstash code contains e.g. mediawiki specific code
• 10:37 valhallasw`cloud: that doesn't seem to be applied... setting has_ganglia: false manually in wikitech hiera
• 10:30 valhallasw`cloud: pulled new changes into puppetmaster to get https://github.com/wikimedia/operations-puppet/commit/4afd23d8e2905a84ef211ad92e8314173eb743ba in
• 10:25 valhallasw`cloud: set Hiera variable "elasticsearch::cluster_name": toolsbeta-logstash-eqiad
• 10:09 valhallasw`cloud: created toolsbeta-logstash to play around with logstash and figure out what we need for tools (phab:T97861)

April 26

• 18:18 valhallasw`cloud: having some issues with puppet-test, so postponing for now
• 17:12 valhallasw`cloud: deploying https://gerrit.wikimedia.org/r/#/c/206118/ on tools-beta using puppet-test

March 31

• 00:27 andrewbogott: shut down toolsbeta-webgrid-03 to conserve resources. It can be restarted when needed.

September 20

• 20:09 andrewbogott_afk: moved toolsbeta-exec-01 and toolsbeta-scfc-icinga-test off of virt1006

July 22

• 11:36 scfc_de: Removed andrewbogott_afk, Coren, petan, YuviPanda from service group admin to prevent further spamming :-)

August 19

• 12:44 petan: rebooting apache it seems to be frozen

August 4

• 23:50 scfc_de: Added scfc_de to local-admin so I don't log myself out again :-)

July 6

• 19:42 petan: rebooting login

June 26

• 08:03 wm-bot: petrb: updating logsplitter

June 24

• 14:47 wm-bot: petrb: rebooting exec-01 to fix the grid weird info
• 13:43 scfc_de: Made scfc root.
• 13:42 scfc_de: Created toolsbeta-puppetmaster.
• 11:09 YuviPanda: Granted yuvipanda root on toolsbeta

June 21

• 13:46 wm-bot: petrb: rebooting all servers

June 17

• 08:31 petan: switching all instances to nfs

June 16

• 15:37 petan: importing sudo policies of tools
• 15:36 petan: importing security groups of tools
• 15:36 petan: blah