Jump to content

Nova Resource:Tools/SAL

From Wikitech

2024-12-13

  • 14:16 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
  • 14:07 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
  • 14:07 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
  • 14:00 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld
  • 09:24 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-68
  • 09:19 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-68
  • 09:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-44
  • 09:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-44
  • 08:27 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-73
  • 08:22 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-73

2024-12-12

  • 10:52 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-5
  • 10:47 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-5

2024-12-06

  • 17:26 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-db-1 (T352206)
  • 17:25 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-db-1 (T352206)
  • 17:24 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-db-3 (T352206)
  • 17:23 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-db-3 (T352206)
  • 07:56 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
  • 07:49 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer

2024-12-05

  • 16:34 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 16:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 14:42 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 14:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 14:06 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
  • 13:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer

2024-12-04

  • 19:33 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
  • 19:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 19:26 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component registry-admission
  • 19:23 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 17:46 andrewbogott: rebooting tools-legacy-redirector-2, many probes failing
  • 17:38 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
  • 17:30 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
  • 17:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 17:03 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
  • 16:54 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 16:47 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 16:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 16:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 15:45 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
  • 15:45 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 15:33 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 15:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 15:26 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
  • 15:23 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 15:18 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
  • 15:11 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 15:11 raymond-ndibe@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component envvars-api
  • 15:09 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 15:09 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
  • 15:00 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 14:46 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
  • 14:45 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 01:31 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:31 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:30 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:30 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:18 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:18 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:17 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:17 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:17 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:16 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:15 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:15 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:14 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 01:12 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component components-api
  • 01:12 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api

2024-12-03

  • 22:11 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-harbor
  • 22:04 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor
  • 22:03 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-harbor
  • 21:56 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-harbor
  • 21:55 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component main
  • 21:55 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component main

2024-11-29

  • 03:43 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 03:37 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 03:37 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 03:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 03:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 03:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api

2024-11-27

  • 18:26 taavi: kubectl sudo rollout restart -n kube-system deployment coredns # update resolv.conf in coredns containers

2024-11-26

  • 10:42 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-control-7
  • 10:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-7
  • 10:36 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-control-7
  • 10:35 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-7
  • 10:34 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-control-7
  • 10:33 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-7
  • 10:32 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-control-7
  • 10:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-7
  • 10:31 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-control-7
  • 10:30 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-7
  • 10:23 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-control-9
  • 10:22 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-9
  • 10:22 dcaro: rebooting k8s-control-9
  • 10:18 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-control-8
  • 10:17 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-control-8
  • 10:17 dcaro: rebooting k8s-control-8
  • 09:15 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-72
  • 09:14 dcaro: restarting tools-k8s-worker-nfs-72
  • 09:14 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-72
  • 09:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-70
  • 09:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-70
  • 09:12 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-50
  • 09:12 dcaro: restarting tools-k8s-worker-nfs-70
  • 09:11 dcaro: restarting tools-k8s-worker-nfs-50
  • 09:11 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-50
  • 09:08 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-17
  • 09:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-17
  • 08:34 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-61
  • 08:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-61
  • 07:30 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all NFS workers (T380827)
  • 06:47 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all NFS workers (T380827)

2024-11-25

  • 13:05 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-cli
  • 12:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli

2024-11-23

  • 07:27 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder (T358225)
  • 07:21 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder (T358225)

2024-11-20

  • 15:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 15:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 14:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
  • 14:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
  • 12:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
  • 12:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
  • 00:22 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission (T362867)
  • 00:16 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission (T362867)

2024-11-19

  • 21:52 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 21:46 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 21:36 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
  • 21:30 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
  • 21:11 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
  • 21:05 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 21:05 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
  • 20:59 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
  • 20:54 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 20:53 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-emailer
  • 20:53 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
  • 20:48 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics
  • 20:38 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api
  • 20:31 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 20:31 raymond-ndibe@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component envvars-api (T362867)
  • 20:31 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api (T362867)
  • 20:30 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api (T362867)
  • 20:28 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api (T362867)
  • 20:17 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico (T362867)
  • 20:12 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T362867)
  • 20:07 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics (T362867)
  • 20:01 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics (T362867)
  • 19:37 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission (T362867)
  • 19:32 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission (T362867)
  • 19:30 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission (T362867)
  • 19:23 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission (T362867)
  • 15:52 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 15:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api

2024-11-18

  • 14:45 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
  • 14:39 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 14:35 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
  • 14:33 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 11:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
  • 11:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer

2024-11-15

  • 14:05 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-db-5.tools.eqiad1.wikimedia.cloud (T352206)
  • 14:04 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-db-5.tools.eqiad1.wikimedia.cloud (T352206)
  • 14:03 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-db' (T352206)
  • 13:57 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-db' (T352206)
  • 13:57 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T352206)
  • 13:57 fnegri@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase (T352206)
  • 13:50 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'tools-db' (T352206)
  • 13:49 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-db' (T352206)

2024-11-14

  • 13:16 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
  • 13:10 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 13:04 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
  • 13:02 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 13:02 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
  • 12:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice

2024-11-12

  • 15:50 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 15:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 10:27 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
  • 10:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 10:11 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
  • 10:08 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice

2024-11-11

  • 16:02 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-db-4.tools.eqiad1.wikimedia.cloud (T352206)
  • 15:58 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-db-4.tools.eqiad1.wikimedia.cloud (T352206)
  • 14:44 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-db-4.tools.eqiad1.wikimedia.cloud (T352206)
  • 14:42 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-db-4.tools.eqiad1.wikimedia.cloud (T352206)
  • 14:41 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-db' (T352206)
  • 14:37 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-db' (T352206)
  • 14:01 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 13:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-11-10

  • 02:47 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T362867)
  • 02:47 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/kube-state-metrics:v2.11.0 (T362867)
  • 02:47 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T362867)

2024-11-06

  • 16:27 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 16:22 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 15:48 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 15:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 10:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-24 (T379139)
  • 10:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-24 (T379139)
  • 07:57 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component tools-webservice
  • 07:52 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 07:20 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 07:14 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-11-05

  • 17:20 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 17:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
  • 09:40 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 09:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 08:38 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 08:32 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics
  • 08:23 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
  • 08:17 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
  • 07:49 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico
  • 07:44 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico

2024-11-04

  • 16:39 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
  • 16:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 16:30 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
  • 16:25 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 16:22 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
  • 16:21 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 15:05 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 14:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 14:47 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-9 from 1.27.16 to 1.28.14 (T362867)
  • 14:46 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-9 from 1.27.16 to 1.28.14 (T362867)
  • 14:46 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-8 from 1.27.16 to 1.28.14 (T362867)
  • 14:45 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-8 from 1.27.16 to 1.28.14 (T362867)
  • 14:45 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-7 from 1.27.16 to 1.28.14 (T362867)
  • 14:44 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-7 from 1.27.16 to 1.28.14 (T362867)
  • 14:42 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-9 from 1.27.16 to 1.28.14 (T362867)
  • 14:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-9 from 1.27.16 to 1.28.14 (T362867)
  • 14:41 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-8 from 1.27.16 to 1.28.14 (T362867)
  • 14:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-8 from 1.27.16 to 1.28.14 (T362867)
  • 14:40 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-76 from 1.27.16 to 1.28.14 (T362867)
  • 14:39 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-76 from 1.27.16 to 1.28.14 (T362867)
  • 14:38 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-75 from 1.27.16 to 1.28.14 (T362867)
  • 14:37 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-75 from 1.27.16 to 1.28.14 (T362867)
  • 14:37 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-74 from 1.27.16 to 1.28.14 (T362867)
  • 14:36 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-74 from 1.27.16 to 1.28.14 (T362867)
  • 14:36 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-73 from 1.27.16 to 1.28.14 (T362867)
  • 14:35 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-73 from 1.27.16 to 1.28.14 (T362867)
  • 14:35 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-72 from 1.27.16 to 1.28.14 (T362867)
  • 14:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-72 from 1.27.16 to 1.28.14 (T362867)
  • 14:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-71 from 1.27.16 to 1.28.14 (T362867)
  • 14:33 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-71 from 1.27.16 to 1.28.14 (T362867)
  • 14:33 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-70 from 1.27.16 to 1.28.14 (T362867)
  • 14:32 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-70 from 1.27.16 to 1.28.14 (T362867)
  • 14:32 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-7 from 1.27.16 to 1.28.14 (T362867)
  • 14:29 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-69 from 1.27.16 to 1.28.14 (T362867)
  • 14:29 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-68 from 1.27.16 to 1.28.14 (T362867)
  • 14:28 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-68 from 1.27.16 to 1.28.14 (T362867)
  • 14:28 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-67 from 1.27.16 to 1.28.14 (T362867)
  • 14:27 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-67 from 1.27.16 to 1.28.14 (T362867)
  • 14:27 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-66 from 1.27.16 to 1.28.14 (T362867)
  • 14:26 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-66 from 1.27.16 to 1.28.14 (T362867)
  • 14:26 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-65 from 1.27.16 to 1.28.14 (T362867)
  • 14:25 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-65 from 1.27.16 to 1.28.14 (T362867)
  • 14:25 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-61 from 1.27.16 to 1.28.14 (T362867)
  • 14:24 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-61 from 1.27.16 to 1.28.14 (T362867)
  • 14:20 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-61 from 1.27.16 to 1.28.14 (T362867)
  • 14:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-61 from 1.27.16 to 1.28.14 (T362867)
  • 14:14 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-58 from 1.27.16 to 1.28.14 (T362867)
  • 14:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-58 from 1.27.16 to 1.28.14 (T362867)
  • 14:08 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-58 from 1.27.16 to 1.28.14 (T362867)
  • 14:02 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-58 from 1.27.16 to 1.28.14 (T362867)
  • 14:02 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-57 from 1.27.16 to 1.28.14 (T362867)
  • 14:01 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-57 from 1.27.16 to 1.28.14 (T362867)
  • 14:01 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-55 from 1.27.16 to 1.28.14 (T362867)
  • 14:00 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-55 from 1.27.16 to 1.28.14 (T362867)
  • 14:00 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-54 from 1.27.16 to 1.28.14 (T362867)
  • 13:59 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-54 from 1.27.16 to 1.28.14 (T362867)
  • 13:59 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-53 from 1.27.16 to 1.28.14 (T362867)
  • 13:57 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-53 from 1.27.16 to 1.28.14 (T362867)
  • 13:57 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-50 from 1.27.16 to 1.28.14 (T362867)
  • 13:56 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-50 from 1.27.16 to 1.28.14 (T362867)
  • 13:56 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-5 from 1.27.16 to 1.28.14 (T362867)
  • 13:55 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.27.16 to 1.28.14 (T362867)
  • 13:55 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-48 from 1.27.16 to 1.28.14 (T362867)
  • 13:54 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-48 from 1.27.16 to 1.28.14 (T362867)
  • 13:54 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-47 from 1.27.16 to 1.28.14 (T362867)
  • 13:53 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-47 from 1.27.16 to 1.28.14 (T362867)
  • 13:53 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-46 from 1.27.16 to 1.28.14 (T362867)
  • 13:52 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-46 from 1.27.16 to 1.28.14 (T362867)
  • 13:51 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-45 from 1.27.16 to 1.28.14 (T362867)
  • 13:50 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-45 from 1.27.16 to 1.28.14 (T362867)
  • 13:50 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-44 from 1.27.16 to 1.28.14 (T362867)
  • 13:49 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-44 from 1.27.16 to 1.28.14 (T362867)
  • 13:49 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-43 from 1.27.16 to 1.28.14 (T362867)
  • 13:48 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-43 from 1.27.16 to 1.28.14 (T362867)
  • 13:48 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-42 from 1.27.16 to 1.28.14 (T362867)
  • 13:47 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-42 from 1.27.16 to 1.28.14 (T362867)
  • 13:47 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-41 from 1.27.16 to 1.28.14 (T362867)
  • 13:46 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-41 from 1.27.16 to 1.28.14 (T362867)
  • 13:46 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-40 from 1.27.16 to 1.28.14 (T362867)
  • 13:44 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-40 from 1.27.16 to 1.28.14 (T362867)
  • 13:44 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-39 from 1.27.16 to 1.28.14 (T362867)
  • 13:43 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-39 from 1.27.16 to 1.28.14 (T362867)
  • 13:43 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-38 from 1.27.16 to 1.28.14 (T362867)
  • 13:42 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-38 from 1.27.16 to 1.28.14 (T362867)
  • 13:42 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-37 from 1.27.16 to 1.28.14 (T362867)
  • 13:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-37 from 1.27.16 to 1.28.14 (T362867)
  • 13:41 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-36 from 1.27.16 to 1.28.14 (T362867)
  • 13:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-36 from 1.27.16 to 1.28.14 (T362867)
  • 13:40 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-35 from 1.27.16 to 1.28.14 (T362867)
  • 13:39 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-35 from 1.27.16 to 1.28.14 (T362867)
  • 13:38 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-34 from 1.27.16 to 1.28.14 (T362867)
  • 13:37 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-34 from 1.27.16 to 1.28.14 (T362867)
  • 13:37 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-33 from 1.27.16 to 1.28.14 (T362867)
  • 13:36 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-33 from 1.27.16 to 1.28.14 (T362867)
  • 13:36 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-32 from 1.27.16 to 1.28.14 (T362867)
  • 13:35 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-32 from 1.27.16 to 1.28.14 (T362867)
  • 13:35 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-3 from 1.27.16 to 1.28.14 (T362867)
  • 13:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-3 from 1.27.16 to 1.28.14 (T362867)
  • 13:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-27 from 1.27.16 to 1.28.14 (T362867)
  • 13:33 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-27 from 1.27.16 to 1.28.14 (T362867)
  • 13:33 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-26 from 1.27.16 to 1.28.14 (T362867)
  • 13:31 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-26 from 1.27.16 to 1.28.14 (T362867)
  • 13:31 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-24 from 1.27.16 to 1.28.14 (T362867)
  • 13:30 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-24 from 1.27.16 to 1.28.14 (T362867)
  • 13:30 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-23 from 1.27.16 to 1.28.14 (T362867)
  • 13:29 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-23 from 1.27.16 to 1.28.14 (T362867)
  • 13:29 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-22 from 1.27.16 to 1.28.14 (T362867)
  • 13:28 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-22 from 1.27.16 to 1.28.14 (T362867)
  • 13:28 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-21 from 1.27.16 to 1.28.14 (T362867)
  • 13:27 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.27.16 to 1.28.14 (T362867)
  • 13:27 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-2 from 1.27.16 to 1.28.14 (T362867)
  • 13:26 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-2 from 1.27.16 to 1.28.14 (T362867)
  • 13:26 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-19 from 1.27.16 to 1.28.14 (T362867)
  • 13:25 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-19 from 1.27.16 to 1.28.14 (T362867)
  • 13:20 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-19 from 1.27.16 to 1.28.14 (T362867)
  • 13:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-19 from 1.27.16 to 1.28.14 (T362867)
  • 13:14 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-17 from 1.27.16 to 1.28.14 (T362867)
  • 13:13 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-17 from 1.27.16 to 1.28.14 (T362867)
  • 13:13 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-16 from 1.27.16 to 1.28.14 (T362867)
  • 13:12 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-16 from 1.27.16 to 1.28.14 (T362867)
  • 13:11 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-14 from 1.27.16 to 1.28.14 (T362867)
  • 13:10 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-14 from 1.27.16 to 1.28.14 (T362867)
  • 13:10 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-13 from 1.27.16 to 1.28.14 (T362867)
  • 13:10 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
  • 13:09 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-13 from 1.27.16 to 1.28.14 (T362867)
  • 13:09 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-12 from 1.27.16 to 1.28.14 (T362867)
  • 13:08 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-12 from 1.27.16 to 1.28.14 (T362867)
  • 13:08 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-11 from 1.27.16 to 1.28.14 (T362867)
  • 13:07 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-11 from 1.27.16 to 1.28.14 (T362867)
  • 13:07 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867)
  • 13:06 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867)
  • 13:04 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 13:04 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
  • 13:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 12:55 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867)
  • 12:49 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867)
  • 12:47 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867)
  • 12:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867)
  • 12:41 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-1 from 1.27.16 to 1.28.14 (T362867)
  • 12:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-1 from 1.27.16 to 1.28.14 (T362867)
  • 12:40 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-108 from 1.27.16 to 1.28.14 (T362867)
  • 12:39 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-108 from 1.27.16 to 1.28.14 (T362867)
  • 12:39 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-107 from 1.27.16 to 1.28.14 (T362867)
  • 12:38 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-107 from 1.27.16 to 1.28.14 (T362867)
  • 12:38 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-106 from 1.27.16 to 1.28.14 (T362867)
  • 12:37 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-106 from 1.27.16 to 1.28.14 (T362867)
  • 12:36 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-105 from 1.27.16 to 1.28.14 (T362867)
  • 12:35 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-105 from 1.27.16 to 1.28.14 (T362867)
  • 12:35 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-103 from 1.27.16 to 1.28.14 (T362867)
  • 12:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-103 from 1.27.16 to 1.28.14 (T362867)
  • 12:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-102 from 1.27.16 to 1.28.14 (T362867)
  • 12:33 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-102 from 1.27.16 to 1.28.14 (T362867)
  • 12:22 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-9 from 1.27.16 to 1.28.14 (T362867)
  • 12:22 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
  • 12:16 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 12:13 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-9 from 1.27.16 to 1.28.14 (T362867)
  • 12:11 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api
  • 12:06 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 12:03 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-8 from 1.27.16 to 1.28.14 (T362867)
  • 11:59 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api
  • 11:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 11:57 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-8 from 1.27.16 to 1.28.14 (T362867)
  • 11:49 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-7 from 1.27.16 to 1.28.14 (T362867)
  • 11:42 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.27.16 to 1.28.14 (T362867)
  • 11:38 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster tools upgrade from 1.27.16 to 1.28.14 (T362867)
  • 11:26 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade from 1.27.16 to 1.28.14 (T362867)
  • 11:19 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 11:14 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
  • 10:56 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 10:50 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 10:42 dcaro: added api.svc.toolforge.org dns record entry
  • 10:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
  • 10:25 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
  • 10:15 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
  • 10:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 09:56 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component registry-admission
  • 09:55 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 09:51 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component registry-admission
  • 09:48 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 09:28 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 09:23 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-10-22

  • 13:05 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-23
  • 13:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-23
  • 12:58 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-33, tools-k8s-woker-nfs-23
  • 12:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-33, tools-k8s-woker-nfs-23
  • 09:05 arturo: restart puppetserver service for T377803

2024-10-16

  • 09:41 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 09:37 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 09:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 09:07 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 09:00 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api

2024-10-15

  • 17:20 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 17:14 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 16:16 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
  • 16:14 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-10-14

  • 09:14 dcaro: migrating pipelineruns stored versions to v1 (T376710)
  • 07:26 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-9
  • 07:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9
  • 07:24 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-9
  • 07:23 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9

2024-10-09

  • 09:27 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
  • 09:17 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-10-08

  • 13:34 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld (T376710)
  • 13:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld (T376710)
  • 12:38 dcaro: tests are passing correctly, upgrade finished, will investigate the increased slowness as a followup
  • 12:27 dcaro: upgrade finished, build actions have become slower than usual (T376710), running tests and investigating
  • 12:02 dcaro: starting toolforge builds-builder upgrade, no downtime expected though some builds might fail to start/list/log/show while the upgrade is in progress T374908
  • 08:26 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 08:24 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
  • 08:24 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-kubeusers
  • 08:24 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-10-04

  • 11:57 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 11:51 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 11:44 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 11:38 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api

2024-10-02

  • 09:11 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component maintain-kubeusers
  • 09:07 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-10-01

  • 10:52 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
  • 10:46 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 10:32 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 10:28 dcaro: updated ci image with latest precommit versions
  • 10:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 09:52 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-admission
  • 09:47 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission

2024-09-30

  • 18:25 taavi: run striker migrations T359428

2024-09-28

  • 00:14 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli
  • 00:07 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli

2024-09-27

  • 23:58 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
  • 23:52 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld

2024-09-26

  • 16:45 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 16:40 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
  • 16:24 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
  • 16:18 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 16:18 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component registry-admission
  • 16:08 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 16:05 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 15:58 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 10:26 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli
  • 10:20 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli
  • 10:12 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-cli
  • 10:05 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component envvars-cli
  • 07:53 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component toolforge-weld
  • 07:46 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component toolforge-weld

2024-09-25

  • 08:00 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-7
  • 07:59 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-7

2024-09-24

  • 22:11 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers (T375157)
  • 22:03 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers (T375157)
  • 21:48 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component kyverno (T359641)
  • 21:41 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component kyverno (T359641)

2024-09-20

  • 20:12 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico (T341066)
  • 20:08 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T341066)
  • 20:08 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component calico (T341066)
  • 20:06 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T341066)
  • 19:36 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component calico (T341066)
  • 19:31 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico (T341066)
  • 17:06 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 17:06 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/calico/pod2daemon-flexvol:v3.28.2 (T359641)
  • 17:05 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 17:04 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 17:04 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/calico/typha:v3.28.2 (T359641)
  • 17:04 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 17:04 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 17:03 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/calico/node:v3.28.2 (T359641)
  • 17:03 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 17:02 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 17:02 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/calico/kube-controllers:v3.28.2 (T359641)
  • 17:02 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 16:59 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 16:59 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/calico/ctl:v3.28.2 (T359641)
  • 16:59 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 16:57 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 16:56 raymond-ndibe@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/calico/cni:v3.28.2 (T359641)
  • 16:56 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 16:54 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/calico/cni:v3.28.2 (T359641)
  • 16:54 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 06:29 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=1)
  • 00:39 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component wmcs-k8s-metrics (T359641)
  • 00:32 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component wmcs-k8s-metrics (T359641)

2024-09-19

  • 23:17 wmbot~raymondndibe@wmf3402: END (ERROR) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=97) (T359641)
  • 23:17 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/metrics-server:v0.7.10 (T359641)
  • 23:17 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 23:12 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 23:11 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/kube-state-metrics:v2.10.1 (T359641)
  • 23:11 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 22:38 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 22:37 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/metrics-server:v0.7.1 (T359641)
  • 22:37 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 22:36 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) (T359641)
  • 22:36 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/metrics-server:v0.7.1 (T359641)
  • 22:36 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 22:35 wmbot~raymondndibe@wmf3402: END (ERROR) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=97) (T359641)
  • 22:35 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/docker-registry.tools.wmflabs.org/metrics-server:v0.7.1 (T359641)
  • 22:35 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 17:47 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-cli (T341066)
  • 17:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-cli (T341066)
  • 17:13 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api (T341066)
  • 17:06 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 16:48 wmbot~raymondndibe@wmf3402: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api (T341066)
  • 16:46 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 16:45 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api
  • 16:43 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 16:38 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 16:26 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 16:10 dcaro: rebooting tools-k8s-worker-nfs-24 it's stuck without network
  • 16:08 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 16:08 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 16:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 16:07 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 16:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 15:28 wmbot~raymondndibe@wmf3402: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api (T341066)
  • 15:27 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 15:19 wmbot~raymondndibe@wmf3402: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api (T341066)
  • 15:18 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 15:08 wmbot~raymondndibe@wmf3402: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component jobs-api (T341066)
  • 15:07 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 15:01 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api (T341066)
  • 14:57 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)
  • 14:56 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api (T341066)
  • 14:50 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api (T341066)

2024-09-17

  • 08:46 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-70 (T359641)
  • 08:43 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-70 (T359641)
  • 08:43 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-k8s-worker-nfs-70.tools.eqiad1.wikimedia.cloud (T359641)
  • 08:41 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-75 (T359641)
  • 08:40 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-k8s-worker-nfs-70.tools.eqiad1.wikimedia.cloud (T359641)
  • 08:40 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-75 (T359641)
  • 08:35 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-k8s-worker-nfs-75.tools.eqiad1.wikimedia.cloud (T359641)
  • 08:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-k8s-worker-nfs-75.tools.eqiad1.wikimedia.cloud (T359641)
  • 03:24 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-9 from 1.26.15 to 1.27.16 (T359641)
  • 03:23 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-9 from 1.26.15 to 1.27.16 (T359641)
  • 03:20 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-8 from 1.26.15 to 1.27.16 (T359641)
  • 03:19 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-8 from 1.26.15 to 1.27.16 (T359641)
  • 03:19 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-7 from 1.26.15 to 1.27.16 (T359641)
  • 03:18 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-7 from 1.26.15 to 1.27.16 (T359641)
  • 03:13 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-64
  • 03:10 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-63
  • 03:08 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-64 (T359641)
  • 03:07 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 03:07 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-76.tools.eqiad1.wikimedia.cloud to the cluster
  • 03:04 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-63 (T359641)
  • 03:00 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 03:00 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-75.tools.eqiad1.wikimedia.cloud to the cluster
  • 02:57 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 02:50 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 02:46 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 02:46 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-74.tools.eqiad1.wikimedia.cloud to the cluster
  • 02:45 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-62
  • 02:45 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-60
  • 02:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-62 (T359641)
  • 02:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-60 (T359641)
  • 02:38 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 02:38 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-73.tools.eqiad1.wikimedia.cloud to the cluster
  • 02:36 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 02:32 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 02:32 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-72.tools.eqiad1.wikimedia.cloud to the cluster
  • 02:29 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 02:24 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 02:24 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-71.tools.eqiad1.wikimedia.cloud to the cluster
  • 02:22 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 02:15 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 02:12 raymond-ndibe@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=97) for a worker-nfs role in the tools cluster (T359641)
  • 02:10 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-6
  • 02:10 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-56
  • 02:08 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 02:08 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-70.tools.eqiad1.wikimedia.cloud to the cluster
  • 02:05 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-6 (T359641)
  • 02:04 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-56 (T359641)
  • 02:02 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-49
  • 02:02 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-31
  • 01:58 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 01:58 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 01:58 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 01:57 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-69.tools.eqiad1.wikimedia.cloud to the cluster
  • 01:57 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-49 (T359641)
  • 01:57 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-31 (T359641)
  • 01:56 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-30
  • 01:54 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-64 (T359641)
  • 01:53 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-29
  • 01:50 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-30 (T359641)
  • 01:49 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-64 (T359641)
  • 01:48 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 01:48 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-29 (T359641)
  • 01:46 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-64 from 1.26.15 to 1.27.16 (T359641)
  • 01:46 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-63 (T359641)
  • 01:45 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-nfs-28
  • 01:42 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 01:42 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-68.tools.eqiad1.wikimedia.cloud to the cluster
  • 01:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-63 (T359641)
  • 01:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-64 from 1.26.15 to 1.27.16 (T359641)
  • 01:40 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-63 from 1.26.15 to 1.27.16 (T359641)
  • 01:40 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-28 (T359641)
  • 01:35 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-62 (T359641)
  • 01:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-63 from 1.26.15 to 1.27.16 (T359641)
  • 01:34 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-62 from 1.26.15 to 1.27.16 (T359641)
  • 01:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-60 (T359641)
  • 01:33 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 01:32 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 01:32 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-67.tools.eqiad1.wikimedia.cloud to the cluster
  • 01:29 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-62 from 1.26.15 to 1.27.16 (T359641)
  • 01:29 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-61 from 1.26.15 to 1.27.16 (T359641)
  • 01:28 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-60 (T359641)
  • 01:28 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-6 (T359641)
  • 01:28 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-61 from 1.26.15 to 1.27.16 (T359641)
  • 01:28 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-60 from 1.26.15 to 1.27.16 (T359641)
  • 01:23 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 01:23 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 01:23 raymond-ndibe@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-66.tools.eqiad1.wikimedia.cloud to the cluster
  • 01:23 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-6 (T359641)
  • 01:22 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-60 from 1.26.15 to 1.27.16 (T359641)
  • 01:22 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-6 from 1.26.15 to 1.27.16 (T359641)
  • 01:21 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-56 (T359641)
  • 01:16 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-6 from 1.26.15 to 1.27.16 (T359641)
  • 01:16 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-57 from 1.26.15 to 1.27.16 (T359641)
  • 01:15 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-56 (T359641)
  • 01:15 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-57 from 1.26.15 to 1.27.16 (T359641)
  • 01:15 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-56 from 1.26.15 to 1.27.16 (T359641)
  • 01:14 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-49 (T359641)
  • 01:14 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster (T359641)
  • 01:09 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-56 from 1.26.15 to 1.27.16 (T359641)
  • 01:09 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-50 from 1.26.15 to 1.27.16 (T359641)
  • 01:09 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-49 (T359641)
  • 01:08 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-50 from 1.26.15 to 1.27.16 (T359641)
  • 01:08 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-49 from 1.26.15 to 1.27.16 (T359641)
  • 01:04 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-31 (T359641)
  • 01:02 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-49 from 1.26.15 to 1.27.16 (T359641)
  • 01:02 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-46 from 1.26.15 to 1.27.16 (T359641)
  • 01:01 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-46 from 1.26.15 to 1.27.16 (T359641)
  • 01:01 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-38 from 1.26.15 to 1.27.16 (T359641)
  • 01:01 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-38 from 1.26.15 to 1.27.16 (T359641)
  • 01:00 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-36 from 1.26.15 to 1.27.16 (T359641)
  • 01:00 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-36 from 1.26.15 to 1.27.16 (T359641)
  • 01:00 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-32 from 1.26.15 to 1.27.16 (T359641)
  • 00:59 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-31 (T359641)
  • 00:59 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-32 from 1.26.15 to 1.27.16 (T359641)
  • 00:59 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-31 from 1.26.15 to 1.27.16 (T359641)
  • 00:58 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-30 (T359641)
  • 00:53 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-31 from 1.26.15 to 1.27.16 (T359641)
  • 00:53 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-30 from 1.26.15 to 1.27.16 (T359641)
  • 00:52 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-29 (T359641)
  • 00:47 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-29 (T359641)
  • 00:47 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-30 from 1.26.15 to 1.27.16 (T359641)
  • 00:47 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-29 from 1.26.15 to 1.27.16 (T359641)
  • 00:47 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-28 (T359641)
  • 00:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-28 (T359641)
  • 00:41 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-29 from 1.26.15 to 1.27.16 (T359641)
  • 00:41 raymond-ndibe@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-28 from 1.26.15 to 1.27.16 (T359641)
  • 00:35 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-28 from 1.26.15 to 1.27.16 (T359641)
  • 00:35 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-27 from 1.26.15 to 1.27.16 (T359641)
  • 00:34 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-27 from 1.26.15 to 1.27.16 (T359641)
  • 00:34 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-26 from 1.26.15 to 1.27.16 (T359641)
  • 00:33 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-26 from 1.26.15 to 1.27.16 (T359641)
  • 00:33 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-22 from 1.26.15 to 1.27.16 (T359641)
  • 00:32 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-22 from 1.26.15 to 1.27.16 (T359641)
  • 00:32 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-21 from 1.26.15 to 1.27.16 (T359641)
  • 00:31 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.26.15 to 1.27.16 (T359641)
  • 00:30 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-60, tools-k8s-worker-nfs-61, tools-k8s-worker-nfs-62, tools-k8s-worker-nfs-63 (T359641)
  • 00:26 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-50, tools-k8s-worker-nfs-56, tools-k8s-worker-nfs-57, tools-k8s-worker-nfs-6 (T359641)
  • 00:10 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-50, tools-k8s-worker-nfs-56, tools-k8s-worker-nfs-57, tools-k8s-worker-nfs-6 (T359641)
  • 00:09 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-38, tools-k8s-worker-nfs-46, tools-k8s-worker-nfs-49, tools-k8s-worker-nfs-50 (T359641)
  • 00:09 raymond-ndibe@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-60, tools-k8s-worker-nfs-61, tools-k8s-worker-nfs-62, tools-k8s-worker-nfs-63 (T359641)
  • 00:04 raymond-ndibe@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-31, tools-k8s-worker-nfs-32, tools-k8s-worker-nfs-33, tools-k8s-worker-nfs-36 (T359641)

2024-09-16

  • 17:56 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-45
  • 17:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-45
  • 17:46 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-6
  • 17:40 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-6

2024-09-13

  • 11:18 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-54 (T374692)
  • 11:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-54 (T374692)
  • 09:42 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-55, tools-k8s-worker-nfs-5, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-14 (T374692)
  • 09:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-55, tools-k8s-worker-nfs-5, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-14 (T374692)
  • 09:12 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-55, tools-k8s-worker-nfs-5, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-14 (T374692)
  • 09:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-55, tools-k8s-worker-nfs-5, tools-k8s-worker-nfs-19, tools-k8s-worker-nfs-14 (T374692)

2024-09-12

  • 12:06 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-33 (T374612)
  • 11:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-16, tools-k8s-worker-nfs-33 (T374612)
  • 11:54 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-23, tools-k8s-worker-16, tools-k8s-worker-nfs-33 (T374612)
  • 11:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-23, tools-k8s-worker-16, tools-k8s-worker-nfs-33 (T374612)
  • 11:42 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-28 (T374612)
  • 11:37 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-28 (T374612)

2024-09-11

  • 10:27 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers

2024-09-09

  • 16:23 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component cert-manager
  • 16:16 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component cert-manager

2024-09-06

  • 08:47 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 08:42 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 08:38 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component jobs-api
  • 08:36 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 07:14 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0)
  • 07:14 sstefanova@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/pause:3.6
  • 07:14 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry

2024-09-05

  • 13:50 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 13:50 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/stakater-reloader:v1.1.0 (T359641)
  • 13:50 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 13:46 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 13:45 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/startupapicheck:v1.15.3 (T359641)
  • 13:45 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 13:41 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) (T359641)
  • 13:41 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/startupapicheck:v1.15.3 (T359641)
  • 13:41 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 13:40 wmbot~raymondndibe@wmf3402: END (FAIL) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=99) (T359641)
  • 13:40 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/startupapicheck:v1.15.3 (T359641)
  • 13:40 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 13:28 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 13:27 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/cainjector:v1.15.3 (T359641)
  • 13:27 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 13:26 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 13:26 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/webhook:v1.15.3 (T359641)
  • 13:26 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)
  • 13:24 wmbot~raymondndibe@wmf3402: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) (T359641)
  • 13:23 wmbot~raymondndibe@wmf3402: Updating container image docker-registry.tools.wmflabs.org/cert-manager/controller:v1.15.3 (T359641)
  • 13:23 wmbot~raymondndibe@wmf3402: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (T359641)

2024-09-04

  • 14:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 14:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 14:03 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 14:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 14:02 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component maintain-kubeusers
  • 13:56 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component maintain-kubeusers
  • 13:41 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 13:37 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 13:36 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 13:35 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 13:07 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 13:03 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
  • 13:02 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 13:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission

2024-09-03

  • 20:19 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 19:53 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-emailer
  • 19:48 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component jobs-emailer
  • 19:36 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 19:29 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 15:46 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component kyverno
  • 15:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno
  • 15:29 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component kyverno
  • 15:22 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component kyverno
  • 14:41 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99)
  • 14:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 14:30 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-admission
  • 14:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-admission
  • 14:06 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=0) (T359641)
  • 14:05 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.28.5 (T359641)
  • 14:05 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.12.5 (T359641)
  • 14:05 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.12.5 (T359641)
  • 14:05 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.12.5 (T359641)
  • 14:04 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.12.5 (T359641)
  • 14:04 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno-cli:v1.12.5 (T359641)
  • 14:04 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.12.5 (T359641)
  • 14:04 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (T359641)
  • 13:56 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
  • 13:56 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=0) (T359641)
  • 13:55 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.28.5 (T359641)
  • 13:54 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.12.5 (T359641)
  • 13:54 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.12.5 (T359641)
  • 13:53 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.12.5 (T359641)
  • 13:53 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.12.5 (T359641)
  • 13:53 wmbot~dcaro@urcuchillay: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.12.5 (T359641)
  • 13:53 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (T359641)
  • 13:50 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 13:23 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 13:17 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 13:04 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 12:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 11:59 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
  • 11:53 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 10:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 10:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 09:57 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
  • 09:51 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
  • 05:15 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-31 from 1.25.16 to 1.26.15
  • 05:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-31 from 1.25.16 to 1.26.15
  • 05:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-30 from 1.25.16 to 1.26.15
  • 05:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-30 from 1.25.16 to 1.26.15
  • 05:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-29 from 1.25.16 to 1.26.15
  • 05:10 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-29 from 1.25.16 to 1.26.15

2024-09-02

  • 14:31 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-108 from 1.25.16 to 1.26.15 (T370249)
  • 14:30 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-108 from 1.25.16 to 1.26.15 (T370249)
  • 14:30 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-107 from 1.25.16 to 1.26.15 (T370249)
  • 14:29 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-107 from 1.25.16 to 1.26.15 (T370249)
  • 14:29 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-106 from 1.25.16 to 1.26.15 (T370249)
  • 14:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-106 from 1.25.16 to 1.26.15 (T370249)
  • 14:28 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-105 from 1.25.16 to 1.26.15 (T370249)
  • 14:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-105 from 1.25.16 to 1.26.15 (T370249)
  • 14:27 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-103 from 1.25.16 to 1.26.15 (T370249)
  • 14:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-103 from 1.25.16 to 1.26.15 (T370249)
  • 14:26 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-102 from 1.25.16 to 1.26.15 (T370249)
  • 14:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-102 from 1.25.16 to 1.26.15 (T370249)
  • 14:20 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-64 from 1.25.16 to 1.26.15 (T370249)
  • 14:19 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-64 from 1.25.16 to 1.26.15 (T370249)
  • 14:17 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-63 from 1.25.16 to 1.26.15 (T370249)
  • 14:17 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-63 from 1.25.16 to 1.26.15 (T370249)
  • 13:33 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-28 from 1.25.16 to 1.26.15
  • 13:32 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-28 from 1.25.16 to 1.26.15
  • 13:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-27 from 1.25.16 to 1.26.15
  • 13:30 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-27 from 1.25.16 to 1.26.15
  • 13:30 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-26 from 1.25.16 to 1.26.15
  • 13:30 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-63 from 1.25.16 to 1.26.15 (T370249)
  • 13:29 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-63 from 1.25.16 to 1.26.15 (T370249)
  • 13:29 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-62 from 1.25.16 to 1.26.15 (T370249)
  • 13:29 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-26 from 1.25.16 to 1.26.15
  • 13:28 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-24 from 1.25.16 to 1.26.15
  • 13:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-62 from 1.25.16 to 1.26.15 (T370249)
  • 13:28 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-61 from 1.25.16 to 1.26.15 (T370249)
  • 13:27 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-24 from 1.25.16 to 1.26.15
  • 13:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-61 from 1.25.16 to 1.26.15 (T370249)
  • 13:27 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-60 from 1.25.16 to 1.26.15 (T370249)
  • 13:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-60 from 1.25.16 to 1.26.15 (T370249)
  • 13:26 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-58 from 1.25.16 to 1.26.15 (T370249)
  • 13:25 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-23 from 1.25.16 to 1.26.15
  • 13:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-58 from 1.25.16 to 1.26.15 (T370249)
  • 13:24 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-57 from 1.25.16 to 1.26.15 (T370249)
  • 13:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-23 from 1.25.16 to 1.26.15
  • 13:23 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-57 from 1.25.16 to 1.26.15 (T370249)
  • 13:23 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-56 from 1.25.16 to 1.26.15 (T370249)
  • 13:23 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-22 from 1.25.16 to 1.26.15
  • 13:22 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-22 from 1.25.16 to 1.26.15
  • 13:22 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-56 from 1.25.16 to 1.26.15 (T370249)
  • 13:22 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-55 from 1.25.16 to 1.26.15 (T370249)
  • 13:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-21 from 1.25.16 to 1.26.15
  • 13:21 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-55 from 1.25.16 to 1.26.15 (T370249)
  • 13:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-54 from 1.25.16 to 1.26.15 (T370249)
  • 13:20 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.25.16 to 1.26.15
  • 13:20 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-54 from 1.25.16 to 1.26.15 (T370249)
  • 13:20 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-53 from 1.25.16 to 1.26.15 (T370249)
  • 13:18 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-53 from 1.25.16 to 1.26.15 (T370249)
  • 13:17 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-51 from 1.25.16 to 1.26.15 (T370249)
  • 13:17 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-51 from 1.25.16 to 1.26.15 (T370249)
  • 13:17 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-50 from 1.25.16 to 1.26.15 (T370249)
  • 13:16 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-20 from 1.25.16 to 1.26.15
  • 13:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-50 from 1.25.16 to 1.26.15 (T370249)
  • 13:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-49 from 1.25.16 to 1.26.15 (T370249)
  • 13:15 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-20 from 1.25.16 to 1.26.15
  • 13:14 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-49 from 1.25.16 to 1.26.15 (T370249)
  • 13:14 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-48 from 1.25.16 to 1.26.15 (T370249)
  • 13:14 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-19 from 1.25.16 to 1.26.15
  • 13:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-48 from 1.25.16 to 1.26.15 (T370249)
  • 13:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-47 from 1.25.16 to 1.26.15 (T370249)
  • 13:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-19 from 1.25.16 to 1.26.15
  • 13:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-17 from 1.25.16 to 1.26.15
  • 13:12 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-47 from 1.25.16 to 1.26.15 (T370249)
  • 13:12 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-46 from 1.25.16 to 1.26.15 (T370249)
  • 13:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-17 from 1.25.16 to 1.26.15
  • 13:11 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-46 from 1.25.16 to 1.26.15 (T370249)
  • 13:11 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-45 from 1.25.16 to 1.26.15 (T370249)
  • 13:10 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-16 from 1.25.16 to 1.26.15
  • 13:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-45 from 1.25.16 to 1.26.15 (T370249)
  • 13:09 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-44 from 1.25.16 to 1.26.15 (T370249)
  • 13:08 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-16 from 1.25.16 to 1.26.15
  • 13:08 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-44 from 1.25.16 to 1.26.15 (T370249)
  • 13:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-43 from 1.25.16 to 1.26.15 (T370249)
  • 13:08 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-14 from 1.25.16 to 1.26.15
  • 13:07 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-43 from 1.25.16 to 1.26.15 (T370249)
  • 13:07 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-42 from 1.25.16 to 1.26.15 (T370249)
  • 13:07 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-14 from 1.25.16 to 1.26.15
  • 13:07 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-13 from 1.25.16 to 1.26.15
  • 13:06 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-42 from 1.25.16 to 1.26.15 (T370249)
  • 13:06 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-41 from 1.25.16 to 1.26.15 (T370249)
  • 13:06 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-13 from 1.25.16 to 1.26.15
  • 13:05 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-12 from 1.25.16 to 1.26.15
  • 13:05 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-41 from 1.25.16 to 1.26.15 (T370249)
  • 13:05 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-40 from 1.25.16 to 1.26.15 (T370249)
  • 13:04 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-12 from 1.25.16 to 1.26.15
  • 13:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-40 from 1.25.16 to 1.26.15 (T370249)
  • 13:04 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-39 from 1.25.16 to 1.26.15 (T370249)
  • 13:03 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-11 from 1.25.16 to 1.26.15
  • 13:02 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-39 from 1.25.16 to 1.26.15 (T370249)
  • 13:02 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-38 from 1.25.16 to 1.26.15 (T370249)
  • 13:01 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-11 from 1.25.16 to 1.26.15
  • 13:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-38 from 1.25.16 to 1.26.15 (T370249)
  • 13:01 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-37 from 1.25.16 to 1.26.15 (T370249)
  • 13:01 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-10 from 1.25.16 to 1.26.15
  • 13:00 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-37 from 1.25.16 to 1.26.15 (T370249)
  • 13:00 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-36 from 1.25.16 to 1.26.15 (T370249)
  • 13:00 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.25.16 to 1.26.15
  • 12:59 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-9 from 1.25.16 to 1.26.15
  • 12:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-36 from 1.25.16 to 1.26.15 (T370249)
  • 12:59 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-35 from 1.25.16 to 1.26.15 (T370249)
  • 12:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-9 from 1.25.16 to 1.26.15
  • 12:57 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-35 from 1.25.16 to 1.26.15 (T370249)
  • 12:57 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-34 from 1.25.16 to 1.26.15 (T370249)
  • 12:57 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-8 from 1.25.16 to 1.26.15
  • 12:56 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-34 from 1.25.16 to 1.26.15 (T370249)
  • 12:56 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-33 from 1.25.16 to 1.26.15 (T370249)
  • 12:56 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-8 from 1.25.16 to 1.26.15
  • 12:55 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-7 from 1.25.16 to 1.26.15
  • 12:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-33 from 1.25.16 to 1.26.15 (T370249)
  • 12:55 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-32 from 1.25.16 to 1.26.15 (T370249)
  • 12:54 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-7 from 1.25.16 to 1.26.15
  • 12:54 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-32 from 1.25.16 to 1.26.15 (T370249)
  • 12:47 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-7 from 1.25.16 to 1.26.15
  • 12:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-7 from 1.25.16 to 1.26.15
  • 12:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-8 from 1.25.16 to 1.26.15
  • 12:43 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-8 from 1.25.16 to 1.26.15
  • 12:41 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-9 from 1.25.16 to 1.26.15
  • 12:40 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-9 from 1.25.16 to 1.26.15
  • 12:35 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-6 from 1.25.16 to 1.26.15
  • 12:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-6 from 1.25.16 to 1.26.15
  • 12:33 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-5 from 1.25.16 to 1.26.15
  • 12:32 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.25.16 to 1.26.15
  • 12:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-3 from 1.25.16 to 1.26.15
  • 12:31 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-3 from 1.25.16 to 1.26.15
  • 12:29 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-2 from 1.25.16 to 1.26.15
  • 12:27 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-2 from 1.25.16 to 1.26.15
  • 12:26 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-1 from 1.25.16 to 1.26.15
  • 12:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-1 from 1.25.16 to 1.26.15
  • 12:24 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-9 from 1.25.16 to 1.26.15
  • 12:12 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-9 from 1.25.16 to 1.26.15
  • 12:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-8 from 1.25.16 to 1.26.15
  • 12:00 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-8 from 1.25.16 to 1.26.15
  • 11:59 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-7 from 1.25.16 to 1.26.15
  • 11:48 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.25.16 to 1.26.15
  • 11:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster tools upgrade from 1.25.16 to 1.26.15
  • 11:43 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade from 1.25.16 to 1.26.15
  • 10:05 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 09:58 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 09:49 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component registry-admission
  • 09:43 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component registry-admission
  • 09:21 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 09:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 09:06 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
  • 09:00 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 08:48 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.toolforge.component.deploy (exit_code=97) for component components-api
  • 08:48 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api

2024-08-29

  • 16:32 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
  • 16:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
  • 08:00 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-nginx
  • 07:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component ingress-nginx

2024-08-27

  • 12:06 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0)
  • 12:06 sstefanova@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/nginx-ingress-controller:v1.11.2
  • 12:06 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry
  • 09:46 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 09:46 wmbot~dcaro@urcuchillay: Added a new k8s worker tools-k8s-worker-108.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:36 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 09:05 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component calico
  • 08:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component calico
  • 08:57 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component calico
  • 08:56 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component calico
  • 08:55 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-104 (T373243)
  • 08:53 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-104 (T373243)
  • 08:38 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-nfs-52 (T373243)
  • 08:37 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-52 (T373243)
  • 08:35 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-nfs-51 (T373243)
  • 08:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-51 (T373243)
  • 08:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-nfs-25 (T373243)
  • 08:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-25 (T373243)
  • 08:31 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-nfs-18 (T373243)
  • 08:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-18 (T373243)
  • 08:29 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-nfs-15 (T373243)
  • 08:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-15 (T373243)
  • 08:26 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-nfs-4 (T373243)
  • 08:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-nfs-4 (T373243)
  • 08:19 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker role in the tools cluster
  • 08:19 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster

2024-08-26

  • 21:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 21:13 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-64.tools.eqiad1.wikimedia.cloud to the cluster
  • 21:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 21:03 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=97) for a worker-nfs role in the tools cluster
  • 21:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 20:23 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 20:23 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-63.tools.eqiad1.wikimedia.cloud to the cluster
  • 20:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 20:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0)
  • 20:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.quota_increase
  • 18:35 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 18:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 17:49 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 17:49 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-62.tools.eqiad1.wikimedia.cloud to the cluster
  • 17:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 17:38 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0)
  • 17:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.quota_increase
  • 17:33 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 17:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 17:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0)
  • 17:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.quota_increase
  • 17:30 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 17:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 17:04 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 17:04 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-61.tools.eqiad1.wikimedia.cloud to the cluster
  • 16:54 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 16:54 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 16:54 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-60.tools.eqiad1.wikimedia.cloud to the cluster
  • 16:42 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 16:30 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 16:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 16:14 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 16:14 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-58.tools.eqiad1.wikimedia.cloud to the cluster
  • 16:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 16:02 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 16:02 wmbot~dcaro@urcuchillay: Added a new k8s worker-nfs tools-k8s-worker-nfs-57.tools.eqiad1.wikimedia.cloud to the cluster
  • 15:50 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:49 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 15:48 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:44 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 15:39 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:38 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=97) for a worker-nfs role in the tools cluster
  • 15:35 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:33 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 15:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:15 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 15:10 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 14:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-4 (T373243)
  • 13:12 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-4, tools-k8s-worker-nfs-15, tools-k8s-worker-nfs-18, tools-k8s-worker-nfs-25, tools-k8s-worker-nfs-51, tools-k8s-worker-nfs-52, tools-k8s-worker-104 (T373243)
  • 13:05 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-4, tools-k8s-worker-nfs-15, tools-k8s-worker-nfs-18, tools-k8s-worker-nfs-25, tools-k8s-worker-nfs-51, tools-k8s-worker-nfs-52, tools-k8s-worker-104 (T373243)
  • 12:53 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-104 (T373243)
  • 12:53 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-104 (T373243)
  • 12:44 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-104 (T373243)
  • 12:42 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-104 (T373243)
  • 11:06 dcaro: manually deleted the coredns pods that had been around for 4d
  • 09:08 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 09:03 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 09:02 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
  • 09:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 09:00 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway
  • 08:58 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 08:18 dcaro: scale up cordens deployment to 4 replicas

2024-08-21

  • 05:44 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
  • 05:38 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component components-api
  • 05:27 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-builder
  • 05:20 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-builder
  • 05:01 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 04:55 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 04:43 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component volume-admission
  • 04:36 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 04:28 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component volume-admission
  • 04:25 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 04:22 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component volume-admission
  • 04:21 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 04:20 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component volume-admission
  • 04:20 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component volume-admission
  • 04:10 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission
  • 04:03 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission
  • 03:49 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 03:42 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 03:33 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 03:28 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 03:19 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api
  • 03:17 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 03:13 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.component.deploy for component builds-api

2024-08-19

  • 22:02 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-24
  • 21:56 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-24
  • 21:52 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-17
  • 21:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-17
  • 21:46 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-17,tools-k8s-worker-nfs-24
  • 21:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-17,tools-k8s-worker-nfs-24

2024-08-15

  • 06:30 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-20
  • 06:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-20

2024-08-13

  • 09:54 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 09:49 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 07:39 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-6
  • 07:33 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-6

2024-08-12

  • 15:33 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 15:27 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway
  • 12:31 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 12:25 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 11:51 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component tools-webservice
  • 11:46 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component tools-webservice
  • 10:30 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api
  • 10:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component jobs-api
  • 09:57 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component api-gateway
  • 09:50 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.component.deploy for component api-gateway

2024-08-08

  • 16:57 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api
  • 16:51 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component envvars-api
  • 16:36 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component builds-api
  • 16:30 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component builds-api
  • 16:11 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component components-api
  • 16:05 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.component.deploy for component components-api

2024-08-06

  • 09:50 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=1)
  • 09:50 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 09:50 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 09:28 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 09:20 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 09:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 09:20 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 09:19 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 09:19 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 09:19 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console

2024-08-05

  • 13:35 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component components-api
  • 13:34 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component components-api
  • 11:42 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 11:42 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 09:18 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 09:18 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 08:38 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 08:38 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-08-01

  • 20:42 bd808: Uncordoned tools-k8s-worker-nfs-55 following reboot
  • 20:40 bd808: Hard reboot of tools-k8s-worker-nfs-55 following drain cookbook run. Stuck pod remained stuck as expected.
  • 20:37 bd808@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-55
  • 20:32 bd808: Draining and rebooting tools-k8s-worker-nfs-55 after reports of stuck pods via irc
  • 20:32 bd808@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-55
  • 15:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component components-api
  • 15:31 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component components-api

2024-07-31

  • 20:37 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
  • 20:36 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 20:26 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component jobs-cli
  • 20:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 16:17 andrewbogott: changing login.tools.wmlabs.org to point to a newer bastion, tools-bastion-12, in response to T371505
  • 11:38 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 11:38 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 11:33 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component components-api
  • 11:33 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component components-api
  • 10:07 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-26, tools-k8s-worker-nfs-43
  • 09:49 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-22, tools-k8s-worker-nfs-26, tools-k8s-worker-nfs-43

2024-07-30

  • 18:08 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
  • 18:06 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 18:06 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
  • 18:05 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 18:02 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component jobs-cli
  • 18:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 18:02 wmbot~raymond@ubuntu: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component jobs-cli
  • 18:01 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 17:59 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component jobs-cli
  • 17:59 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 17:49 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component jobs-cli
  • 17:49 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 17:40 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component jobs-cli
  • 17:39 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 17:37 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 17:36 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 16:34 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-23
  • 16:28 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-23

2024-07-29

  • 18:24 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 18:23 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 18:06 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 18:05 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 16:24 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 16:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 14:05 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.rebuild_dbinstance (exit_code=0)
  • 14:03 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.rebuild_dbinstance
  • 13:19 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-cli
  • 13:18 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-cli
  • 12:08 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-cli
  • 12:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli
  • 12:01 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-cli
  • 12:00 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-cli

2024-07-25

  • 15:19 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 15:19 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 08:37 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 08:37 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics

2024-07-24

  • 09:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
  • 09:21 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
  • 08:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
  • 08:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
  • 07:07 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component ingress-admission
  • 06:57 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission

2024-07-23

  • 15:04 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 15:04 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
  • 13:49 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 13:49 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 12:20 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 12:20 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 12:15 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 12:14 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 12:08 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:08 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 08:01 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 08:00 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-07-22

  • 17:42 dcaro: moved the apt repo to service endpoint deb.svc.toolforge.org
  • 17:39 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-3
  • 17:38 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-3
  • 17:03 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 17:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 17:00 dcaro: moving the toolforge apt repo to tools-services-06
  • 16:55 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-services-06.tools.eqiad1.wikimedia.cloud
  • 16:53 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-services-06.tools.eqiad1.wikimedia.cloud
  • 09:59 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 09:58 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 09:43 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 09:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-07-19

  • 12:46 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0)
  • 12:46 sstefanova@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/kube-state-metrics:v2.9.2
  • 12:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry
  • 10:03 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0)
  • 10:02 sstefanova@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/nginx-ingress-controller:v1.9.6
  • 10:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry

2024-07-18

  • 14:39 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 14:39 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 08:49 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 08:49 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-07-17

  • 14:50 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:50 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 11:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 11:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 11:12 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
  • 11:12 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 10:44 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 10:44 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 10:25 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 10:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 10:13 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 10:13 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 09:07 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 09:07 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 08:26 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
  • 08:26 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx

2024-07-16

  • 15:03 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 15:03 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-7 from 1.24.17 to 1.25.16
  • 14:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-7 from 1.24.17 to 1.25.16
  • 14:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-8 from 1.24.17 to 1.25.16
  • 14:10 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-8 from 1.24.17 to 1.25.16
  • 14:09 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-9 from 1.24.17 to 1.25.16
  • 14:08 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-9 from 1.24.17 to 1.25.16
  • 11:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 11:35 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 11:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 11:31 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-28 from 1.24.17 to 1.25.16
  • 11:30 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-28 from 1.24.17 to 1.25.16
  • 11:30 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-27 from 1.24.17 to 1.25.16
  • 11:28 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-27 from 1.24.17 to 1.25.16
  • 11:28 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-26 from 1.24.17 to 1.25.16
  • 11:27 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-26 from 1.24.17 to 1.25.16
  • 11:26 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-25 from 1.24.17 to 1.25.16
  • 11:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 11:25 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-25 from 1.24.17 to 1.25.16
  • 11:24 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-24 from 1.24.17 to 1.25.16
  • 11:23 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 11:23 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-24 from 1.24.17 to 1.25.16
  • 11:23 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-23 from 1.24.17 to 1.25.16
  • 11:22 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 11:22 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-23 from 1.24.17 to 1.25.16
  • 11:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-22 from 1.24.17 to 1.25.16
  • 11:20 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-22 from 1.24.17 to 1.25.16
  • 11:16 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-107 from 1.24.17 to 1.25.16
  • 11:15 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-107 from 1.24.17 to 1.25.16
  • 11:15 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-106 from 1.24.17 to 1.25.16
  • 11:14 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-106 from 1.24.17 to 1.25.16
  • 11:13 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-105 from 1.24.17 to 1.25.16
  • 11:12 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-105 from 1.24.17 to 1.25.16
  • 11:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-21 from 1.24.17 to 1.25.16
  • 11:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.24.17 to 1.25.16
  • 11:10 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-nfs-worker-21 from 1.24.17 to 1.25.16
  • 11:10 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-nfs-worker-21 from 1.24.17 to 1.25.16
  • 11:08 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-21
  • 11:02 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-21
  • 10:59 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-104 from 1.24.17 to 1.25.16
  • 10:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-104 from 1.24.17 to 1.25.16
  • 10:57 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-103 from 1.24.17 to 1.25.16
  • 10:57 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-21 from 1.24.17 to 1.25.16
  • 10:56 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-103 from 1.24.17 to 1.25.16
  • 10:55 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-102 from 1.24.17 to 1.25.16
  • 10:54 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-102 from 1.24.17 to 1.25.16
  • 10:53 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-56 from 1.24.17 to 1.25.16
  • 10:52 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-56 from 1.24.17 to 1.25.16
  • 10:51 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-55 from 1.24.17 to 1.25.16
  • 10:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.24.17 to 1.25.16
  • 10:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-20 from 1.24.17 to 1.25.16
  • 10:50 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-55 from 1.24.17 to 1.25.16
  • 10:50 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-54 from 1.24.17 to 1.25.16
  • 10:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-20 from 1.24.17 to 1.25.16
  • 10:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-19 from 1.24.17 to 1.25.16
  • 10:49 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-54 from 1.24.17 to 1.25.16
  • 10:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-19 from 1.24.17 to 1.25.16
  • 10:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-18 from 1.24.17 to 1.25.16
  • 10:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-18 from 1.24.17 to 1.25.16
  • 10:48 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-17 from 1.24.17 to 1.25.16
  • 10:47 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-53 from 1.24.17 to 1.25.16
  • 10:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-17 from 1.24.17 to 1.25.16
  • 10:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-16 from 1.24.17 to 1.25.16
  • 10:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-53 from 1.24.17 to 1.25.16
  • 10:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-16 from 1.24.17 to 1.25.16
  • 10:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-15 from 1.24.17 to 1.25.16
  • 10:45 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-52 from 1.24.17 to 1.25.16
  • 10:44 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-15 from 1.24.17 to 1.25.16
  • 10:44 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-14 from 1.24.17 to 1.25.16
  • 10:44 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-52 from 1.24.17 to 1.25.16
  • 10:43 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-14 from 1.24.17 to 1.25.16
  • 10:43 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-13 from 1.24.17 to 1.25.16
  • 10:43 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-51 from 1.24.17 to 1.25.16
  • 10:42 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-13 from 1.24.17 to 1.25.16
  • 10:42 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-12 from 1.24.17 to 1.25.16
  • 10:42 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-51 from 1.24.17 to 1.25.16
  • 10:41 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-50 from 1.24.17 to 1.25.16
  • 10:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-12 from 1.24.17 to 1.25.16
  • 10:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-11 from 1.24.17 to 1.25.16
  • 10:40 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-50 from 1.24.17 to 1.25.16
  • 10:40 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-49 from 1.24.17 to 1.25.16
  • 10:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-11 from 1.24.17 to 1.25.16
  • 10:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-10 from 1.24.17 to 1.25.16
  • 10:39 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-49 from 1.24.17 to 1.25.16
  • 10:39 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.24.17 to 1.25.16
  • 10:39 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-9 from 1.24.17 to 1.25.16
  • 10:39 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-48 from 1.24.17 to 1.25.16
  • 10:38 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-9 from 1.24.17 to 1.25.16
  • 10:38 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-8 from 1.24.17 to 1.25.16
  • 10:38 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-48 from 1.24.17 to 1.25.16
  • 10:37 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-47 from 1.24.17 to 1.25.16
  • 10:37 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-8 from 1.24.17 to 1.25.16
  • 10:37 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-7 from 1.24.17 to 1.25.16
  • 10:36 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-47 from 1.24.17 to 1.25.16
  • 10:35 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-7 from 1.24.17 to 1.25.16
  • 10:35 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-6 from 1.24.17 to 1.25.16
  • 10:35 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-46 from 1.24.17 to 1.25.16
  • 10:34 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-6 from 1.24.17 to 1.25.16
  • 10:34 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-46 from 1.24.17 to 1.25.16
  • 10:34 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-45 from 1.24.17 to 1.25.16
  • 10:32 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-45 from 1.24.17 to 1.25.16
  • 10:32 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-44 from 1.24.17 to 1.25.16
  • 10:31 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-44 from 1.24.17 to 1.25.16
  • 10:31 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-43 from 1.24.17 to 1.25.16
  • 10:29 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-43 from 1.24.17 to 1.25.16
  • 10:29 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-42 from 1.24.17 to 1.25.16
  • 10:28 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-42 from 1.24.17 to 1.25.16
  • 10:27 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-41 from 1.24.17 to 1.25.16
  • 10:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-41 from 1.24.17 to 1.25.16
  • 10:26 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-40 from 1.24.17 to 1.25.16
  • 10:25 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-40 from 1.24.17 to 1.25.16
  • 10:24 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-39 from 1.24.17 to 1.25.16
  • 10:23 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-39 from 1.24.17 to 1.25.16
  • 10:23 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-38 from 1.24.17 to 1.25.16
  • 10:22 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-38 from 1.24.17 to 1.25.16
  • 10:21 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-37 from 1.24.17 to 1.25.16
  • 10:20 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-6 from 1.24.17 to 1.25.16
  • 10:20 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-37 from 1.24.17 to 1.25.16
  • 10:19 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-36 from 1.24.17 to 1.25.16
  • 10:18 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-36 from 1.24.17 to 1.25.16
  • 10:17 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-35 from 1.24.17 to 1.25.16
  • 10:16 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-35 from 1.24.17 to 1.25.16
  • 10:16 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-34 from 1.24.17 to 1.25.16
  • 10:15 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-34 from 1.24.17 to 1.25.16
  • 10:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-6 from 1.24.17 to 1.25.16
  • 10:14 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-5 from 1.24.17 to 1.25.16
  • 10:14 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
  • 10:14 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-33 from 1.24.17 to 1.25.16
  • 10:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission
  • 10:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.24.17 to 1.25.16
  • 10:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-3 from 1.24.17 to 1.25.16
  • 10:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-33 from 1.24.17 to 1.25.16
  • 10:12 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-32 from 1.24.17 to 1.25.16
  • 10:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-3 from 1.24.17 to 1.25.16
  • 10:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-2 from 1.24.17 to 1.25.16
  • 10:11 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-32 from 1.24.17 to 1.25.16
  • 10:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-2 from 1.24.17 to 1.25.16
  • 10:11 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-31 from 1.24.17 to 1.25.16
  • 10:11 aborrero@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=97) for node tools-k8s-worker-nfs-6 from 1.24.17 to 1.25.16
  • 10:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-6 from 1.24.17 to 1.25.16
  • 10:10 aborrero@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=97) for node tools-k8s-worker-nfs-5 from 1.24.17 to 1.25.16
  • 10:10 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.24.17 to 1.25.16
  • 10:10 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-4 from 1.24.17 to 1.25.16
  • 10:10 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-31 from 1.24.17 to 1.25.16
  • 10:10 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-30 from 1.24.17 to 1.25.16
  • 10:09 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-4 from 1.24.17 to 1.25.16
  • 10:08 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-30 from 1.24.17 to 1.25.16
  • 10:08 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-29 from 1.24.17 to 1.25.16
  • 10:07 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-29 from 1.24.17 to 1.25.16
  • 09:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-1 from 1.24.17 to 1.25.16
  • 09:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-1 from 1.24.17 to 1.25.16
  • 09:50 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-1 from 1.24.17 to 1.25.16
  • 09:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-1 from 1.24.17 to 1.25.16
  • 09:48 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-9 from 1.24.17 to 1.25.16
  • 09:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-9 from 1.24.17 to 1.25.16
  • 09:39 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-8 from 1.24.17 to 1.25.16
  • 09:28 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-8 from 1.24.17 to 1.25.16
  • 09:17 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-7 from 1.24.17 to 1.25.16
  • 09:10 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.24.17 to 1.25.16
  • 09:07 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster tools upgrade from 1.24.17 to 1.25.16
  • 09:06 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade from 1.24.17 to 1.25.16
  • 08:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
  • 08:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission

2024-07-15

  • 14:42 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:42 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 11:40 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 11:40 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 08:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 08:02 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2024-07-11

  • 17:49 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 17:49 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 13:49 dcaro: deploy toolforge-jobs-framework 16.0.13 (T369573)
  • 11:55 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
  • 11:55 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission

2024-07-10

  • 17:09 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 17:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 16:57 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 16:57 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 16:01 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 16:01 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 15:16 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 15:16 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:10 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 10:10 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-07-09

  • 14:21 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 14:21 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 14:19 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:18 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-07-08

  • 20:22 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-37
  • 20:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-37
  • 14:09 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 14:08 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 13:57 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-elastic-3
  • 13:57 andrew@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-elastic-3
  • 13:57 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-elastic-2
  • 13:56 andrew@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-elastic-2
  • 13:56 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-elastic-1
  • 13:56 andrew@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-elastic-1
  • 13:36 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 13:36 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 13:20 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 13:20 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
  • 12:49 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
  • 12:49 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
  • 12:00 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 11:59 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 08:46 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 08:46 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-07-05

  • 12:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 12:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 12:34 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 12:34 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 12:29 wmbot~arturo@nostromo: END (PASS) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=0)
  • 12:29 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4
  • 12:29 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7
  • 12:28 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7
  • 12:28 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7
  • 12:28 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7
  • 12:28 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7
  • 12:27 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 12:27 wmbot~arturo@nostromo: END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99)
  • 12:26 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7
  • 12:26 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7
  • 12:26 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7
  • 12:26 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7
  • 12:26 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7
  • 12:26 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 12:23 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0)
  • 12:23 sstefanova@cloudcumin1001: Updating container image docker-registry.tools.wmflabs.org/kube-state-metrics:v2.7.0
  • 12:23 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry
  • 11:29 wmbot~arturo@nostromo: END (PASS) - Cookbook wmcs.toolforge.k8s.image.copy_to_registry (exit_code=0) copy image from bitnami/kubectl:1.26.4 to docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4
  • 11:28 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4
  • 11:28 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.image.copy_to_registry copy image from bitnami/kubectl:1.26.4 to docker-registry.tools.wmflabs.org/bitnami-kubectl:1.26.4
  • 01:47 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 01:46 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-07-04

  • 17:09 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 17:09 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 12:57 arturo: updating kubelet flags T355881
  • 12:00 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 12:00 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 11:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 11:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 09:43 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 09:43 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 09:34 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 09:34 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 07:54 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 07:53 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-07-03

  • 12:25 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:25 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 10:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 10:21 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 09:59 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 09:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2024-07-02

  • 17:16 andrewbogott: draining (I hope) tools-elastic-3 and tools-elastic-1 for T311905
  • 17:07 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 17:07 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 16:55 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 16:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 15:01 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 15:01 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 11:53 arturo: cleanup kubeadm configmap from TTLAfterFinished settings (T349197)
  • 11:51 arturo: remove --feature-gates=TTLAfterFinished=true from kube-controller-manager static pod definition (T349197)
  • 10:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 10:54 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 09:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 09:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 09:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component cert-manager
  • 09:22 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component cert-manager
  • 09:10 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 09:10 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-07-01

  • 15:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 15:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 14:59 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 14:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
  • 14:42 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 13:21 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
  • 13:21 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
  • 13:06 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 13:06 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission

2024-06-28

  • 11:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 11:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 09:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 09:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 09:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 09:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 09:38 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 09:37 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 09:28 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 09:28 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway

2024-06-27

  • 16:49 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-etcd-23
  • 16:44 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-etcd-23
  • 16:22 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-db-1
  • 16:21 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-db-1
  • 15:49 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server tools-db-1
  • 15:49 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-db-1
  • 15:48 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-db-3
  • 15:46 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-db-3
  • 15:40 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-etcd-24
  • 15:37 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-etcd-24
  • 15:36 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-etcd-22
  • 15:33 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-etcd-22
  • 15:03 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component cert-manager
  • 15:03 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component cert-manager
  • 14:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
  • 14:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx
  • 11:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 11:02 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:02 arturo: drop all PSP definitions for all accounts (T368142)
  • 10:02 arturo: disabled PodSecurityPolicy admission plugin from kubeadm configmap (T368142)
  • 09:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 08:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 08:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-26

  • 11:40 taavi: update pywikibot image to 9.2 T363631
  • 10:43 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:43 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:18 arturo: deploying toolforge-webservice 0.103.9 (T368463)
  • 09:18 arturo: setting kyverno policies to Enforce (T368141)
  • 09:17 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:17 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 08:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-29
  • 08:01 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-29

2024-06-25

  • 21:50 bd808: Live hacked /usr/lib/python3/dist-packages/toolsws/backends/kubernetes.py on login-buster.toolforge.org to remove the `-> dict[str, Any]` type annotations causing T368463
  • 12:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-104
  • 12:30 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-104
  • 12:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-103
  • 12:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-104
  • 12:28 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-104
  • 12:28 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-103
  • 12:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-102
  • 12:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-103
  • 12:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-103
  • 12:26 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-102
  • 12:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-56
  • 12:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-102
  • 12:25 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-102
  • 12:24 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-56
  • 12:24 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-55
  • 12:23 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-55
  • 12:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-54
  • 12:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-56
  • 12:21 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-56
  • 12:21 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-54
  • 12:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-53
  • 12:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-55
  • 12:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-55
  • 12:20 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-53
  • 12:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-54
  • 12:16 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server tools-k8s-worker-nfs-52
  • 12:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-54
  • 12:16 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-52
  • 12:14 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 12:14 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 12:13 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-51
  • 12:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-53
  • 12:11 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-51
  • 12:11 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-53
  • 11:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-50
  • 11:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-52
  • 11:56 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-50
  • 11:56 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server tools-k8s-worker-50
  • 11:56 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-50
  • 11:56 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-52
  • 11:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-51
  • 11:51 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server tools-k8s-worker-50
  • 11:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-51
  • 11:51 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-50
  • 11:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-50
  • 11:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-50
  • 11:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-proxy-7
  • 11:10 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-proxy-7
  • 11:09 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.migrate_floating_ip (exit_code=0) for address 185.15.56.11 to server 'tools-proxy-8'
  • 11:09 taavi@cloudcumin1001: START - Cookbook wmcs.vps.migrate_floating_ip for address 185.15.56.11 to server 'tools-proxy-8'
  • 09:44 arturo: deploy toolforge-webservice 0.103.8 (T362050)
  • 09:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-haproxy-6
  • 09:30 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-haproxy-6
  • 09:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-control-9
  • 09:28 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-control-9
  • 09:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 09:23 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 09:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-ingress-9
  • 09:21 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-ingress-9
  • 08:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-49
  • 08:48 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-49
  • 08:48 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-48
  • 08:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-49
  • 08:47 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-48
  • 08:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-49
  • 08:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-47
  • 08:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-48
  • 08:45 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-48
  • 08:45 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-47
  • 08:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-46
  • 08:44 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-46
  • 08:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-45
  • 08:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-47
  • 08:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-47
  • 08:42 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-45
  • 08:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-44
  • 08:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-46
  • 08:42 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-46
  • 08:40 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-44
  • 08:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-45
  • 08:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-45
  • 08:40 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server tools-k8s-worker-nfs-43
  • 08:39 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-43
  • 08:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-42
  • 08:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-44
  • 08:38 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-44
  • 08:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-43
  • 08:36 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-43
  • 08:36 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-42
  • 08:13 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-42
  • 08:08 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-42
  • 08:07 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-42
  • 08:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-41
  • 08:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-42
  • 08:02 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-41
  • 08:01 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-40
  • 07:59 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-40
  • 07:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-39
  • 07:58 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-41
  • 07:58 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-41
  • 07:58 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-39
  • 07:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-38
  • 07:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-40
  • 07:56 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-40
  • 07:56 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-38
  • 07:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-37
  • 07:55 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-39
  • 07:55 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-39
  • 07:55 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-37
  • 07:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-36
  • 07:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-38
  • 07:53 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-38
  • 07:53 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-36
  • 07:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-35
  • 07:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-37
  • 07:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-37
  • 07:40 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-35
  • 07:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-34
  • 07:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-36
  • 07:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-36
  • 07:37 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-34
  • 07:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-35
  • 07:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-33
  • 07:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-35
  • 07:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-34
  • 07:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-34
  • 07:31 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-33
  • 07:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-33
  • 07:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-33

2024-06-24

  • 20:56 andrewbogott: rebooting tools-k8s-worker-nfs-36; it has lots of stuck processes which somehow didn't get unstuck when we did the post-nfs-migration reboots.
  • 15:55 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-32
  • 15:53 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-32
  • 15:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-31
  • 15:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-32
  • 15:51 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-31
  • 15:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-32
  • 15:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-30
  • 15:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-31
  • 15:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-31
  • 15:48 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-30
  • 15:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-29
  • 15:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-30
  • 15:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-30
  • 15:46 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-29
  • 15:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-28
  • 15:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-29
  • 15:45 arturo: deploy toolforge-webservice 0.103.7 (T362050)
  • 15:44 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-29
  • 15:44 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-28
  • 15:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-27
  • 15:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-28
  • 15:42 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-27
  • 15:42 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-28
  • 15:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-27
  • 15:32 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-27
  • 15:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all NFS workers
  • 14:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-sgebastion-10
  • 14:37 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-sgebastion-10
  • 14:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-bastion-13
  • 14:34 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-bastion-13
  • 14:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-bastion-12
  • 14:30 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-bastion-12
  • 14:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all NFS workers
  • 14:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-nfs-2
  • 14:24 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-nfs-2
  • 13:57 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=99) for server tools-nfs-2
  • 13:57 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-nfs-2
  • 13:50 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs (exit_code=0) for server tbd
  • 13:43 andrew@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_dbinstance_to_ovs for server tbd
  • 13:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-26
  • 13:41 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-26
  • 13:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-25
  • 13:39 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-25
  • 13:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-26
  • 13:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-24
  • 13:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-26
  • 13:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-25
  • 13:37 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-24
  • 13:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-25
  • 13:35 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-23
  • 13:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-24
  • 13:34 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-23
  • 13:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-24
  • 13:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-22
  • 13:29 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-22
  • 13:28 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-21
  • 13:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-23
  • 13:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-23
  • 13:26 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-21
  • 13:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-20
  • 13:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-22
  • 13:24 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-22
  • 13:24 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-20
  • 13:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-21
  • 13:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-19
  • 13:23 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-21
  • 13:21 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-19
  • 13:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-18
  • 13:19 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-18
  • 13:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-20
  • 13:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-17
  • 13:18 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-20
  • 13:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-19
  • 13:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-19
  • 13:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-18
  • 13:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-18
  • 13:16 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-17
  • 13:16 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-17
  • 13:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-17
  • 13:15 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-17
  • 13:15 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-17
  • 13:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-16
  • 13:09 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-16
  • 12:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-15
  • 12:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-16
  • 12:58 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-16
  • 12:58 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-15
  • 12:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-14
  • 12:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-15
  • 12:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-15
  • 12:51 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-14
  • 12:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-13
  • 12:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-14
  • 12:45 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-14
  • 12:45 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-13
  • 12:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-12
  • 12:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-13
  • 12:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-13
  • 12:37 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-12
  • 12:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-11
  • 12:35 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-12
  • 12:35 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-11
  • 12:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-12
  • 12:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-prometheus-7
  • 12:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-11
  • 12:32 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-11
  • 12:32 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-prometheus-7
  • 12:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-control-8
  • 12:24 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-control-8
  • 12:15 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-ingress-8
  • 12:13 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-ingress-8
  • 12:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 12:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 12:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-static-15
  • 12:05 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-static-15
  • 12:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-acme-chief-4
  • 12:02 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-acme-chief-4
  • 12:00 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-10
  • 11:58 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=97) for node tools-k8s-worker-nfs-10
  • 11:58 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-10
  • 11:57 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-10
  • 11:56 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-10
  • 11:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-10
  • 11:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 11:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 11:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-9
  • 11:42 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-9
  • 11:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-8
  • 11:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-9
  • 11:40 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-8
  • 11:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-9
  • 11:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-8
  • 11:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-8
  • 11:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-7
  • 11:37 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-8
  • 11:37 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-7
  • 11:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-8
  • 11:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-7
  • 11:36 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-7
  • 11:35 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-6
  • 11:33 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-6
  • 11:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-5
  • 11:32 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-5
  • 11:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-6
  • 11:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-4
  • 11:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-6
  • 11:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-5
  • 11:30 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-4
  • 11:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-5
  • 11:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-4
  • 11:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-4
  • 11:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-3
  • 11:25 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-3
  • 11:24 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-2
  • 11:23 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-2
  • 11:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-1
  • 11:21 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-1
  • 11:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-3
  • 11:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-3
  • 11:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-2
  • 11:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-2
  • 11:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-1
  • 11:19 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-1
  • 11:17 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-1
  • 11:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-1
  • 10:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-redis-5
  • 10:28 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-redis-5
  • 10:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-docker-registry-7
  • 10:19 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-docker-registry-7
  • 10:17 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:17 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:13 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-ingress-7
  • 10:11 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-worker-nfs-43
  • 10:11 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-ingress-7
  • 10:09 fnegri@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-worker-nfs-43
  • 10:08 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-control-7
  • 10:06 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-control-7
  • 10:04 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-redis-7
  • 10:03 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-43
  • 10:02 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-redis-7
  • 10:01 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-redis-6
  • 09:59 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-redis-6
  • 09:58 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-43
  • 09:53 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-cumin-1
  • 09:52 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-cumin-1
  • 09:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-k8s-haproxy-5
  • 09:50 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-k8s-haproxy-5
  • 09:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-harbor-1
  • 09:47 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-harbor-1
  • 09:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 09:46 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-107.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-prometheus-6
  • 09:39 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-prometheus-6
  • 09:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 09:35 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-puppetserver-01
  • 09:34 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-puppetserver-01
  • 09:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-puppetdb-2
  • 09:32 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-puppetdb-2
  • 09:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-mail-4
  • 09:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 09:30 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-106.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:30 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-mail-4
  • 09:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-legacy-redirector-2
  • 09:28 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-legacy-redirector-2
  • 09:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-imagebuilder-2
  • 09:26 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-imagebuilder-2
  • 09:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-proxy-8
  • 09:24 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-proxy-8
  • 09:24 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-services-05
  • 09:23 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-services-05
  • 09:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-package-builder-04
  • 09:21 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-package-builder-04
  • 09:21 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 09:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-docker-registry-8
  • 09:20 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker role in the tools cluster
  • 09:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 09:19 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-docker-registry-8
  • 09:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.migrate_server_to_ovs (exit_code=0) for server tools-checker-5
  • 09:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 09:18 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-105.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:18 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.migrate_server_to_ovs for server tools-checker-5
  • 09:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 09:08 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker role in the tools cluster
  • 09:07 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster

2024-06-20

  • 13:09 arturo: re-deploy kyverno T368044
  • 12:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 12:55 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 09:19 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:19 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 09:08 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:08 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-19

  • 10:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 10:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 10:11 arturo: merging k8s HAproxy change https://gerrit.wikimedia.org/r/c/operations/puppet/+/1047113
  • 04:18 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 04:17 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 04:16 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 04:15 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-06-14

  • 14:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 14:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:38 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 08:15 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 08:14 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 07:35 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 07:35 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-06-12

  • 19:41 bd808: Rebuilding all shared Docker containers. This will among other things apply the fix for T367345.
  • 17:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 17:21 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 17:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 17:19 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 16:52 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 16:28 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:28 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 15:24 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 15:24 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno
  • 15:03 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 15:03 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 13:52 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 13:52 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 13:45 taavi: hard reboot tools-k8s-control-7
  • 12:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-11

  • 17:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all NFS workers
  • 16:42 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 16:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 16:41 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 16:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 16:38 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 16:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 15:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all NFS workers
  • 15:50 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for all NFS workers
  • 15:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all NFS workers
  • 11:35 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 11:35 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 11:12 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 11:12 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:57 dcaro: cleaning old maintain-kubeusers configmaps
  • 10:45 dcaro: cleaning up old resourcequotas

2024-06-10

  • 09:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component kyverno
  • 09:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component kyverno

2024-06-07

  • 10:10 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 10:09 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 09:59 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 09:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-06-06

  • 14:21 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:21 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 14:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:06 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:05 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-06-05

  • 16:05 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:05 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 13:27 dcaro: deploying toolforge-webservice 0.103.6
  • 12:58 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 12:58 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 08:44 dcaro: deploying toolforge-jobs-framework-cli 16.0.10 on tools-bastion-13
  • 08:41 dcaro: deploying toolforge-jobs-framework-cli 16.0.10 on tools-bastion-12

2024-06-04

  • 16:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:47 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 12:47 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 12:19 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:19 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:09 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:08 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:32 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:32 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 09:26 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:26 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 08:12 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 08:12 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-06-03

  • 16:26 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 16:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 16:05 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:04 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 16:01 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:01 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 16:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 15:58 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 15:57 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 14:11 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:16 wmbot~arturo@nostromo: END (PASS) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=0)
  • 10:15 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-reports-controller:v1.10.7
  • 10:15 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-cleanup-controller:v1.10.7
  • 10:14 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-background-controller:v1.10.7
  • 10:14 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyvernopre:v1.10.7
  • 10:14 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7
  • 10:14 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 10:13 wmbot~arturo@nostromo: END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99)
  • 10:13 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7
  • 10:13 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 09:37 wmbot~arturo@nostromo: END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99)
  • 09:37 wmbot~arturo@nostromo: Updating container image docker-registry.tools.wmflabs.org/toolforge-kyverno-kyverno:v1.10.7
  • 09:37 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 09:29 wmbot~arturo@nostromo: END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99)
  • 09:29 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 09:29 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 09:29 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 09:29 wmbot~arturo@nostromo: END (FAIL) - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry (exit_code=99)
  • 09:28 wmbot~arturo@nostromo: START - Cookbook wmcs.toolforge.k8s.kyverno.copy_images_to_registry
  • 09:13 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 09:13 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 08:43 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 08:43 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission

2024-05-29

  • 16:14 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 16:13 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 02:59 wmbot~raymond@ubuntu: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component envvars-api
  • 02:59 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-05-28

  • 10:44 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 10:44 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-05-27

  • 15:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 15:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 09:22 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-9
  • 09:21 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9

2024-05-25

  • 21:33 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 21:32 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 20:38 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 20:37 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-05-23

  • 13:22 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 13:21 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-05-22

  • 16:36 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-9
  • 16:36 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9

2024-05-15

  • 14:17 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-9 (T364822)
  • 14:16 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9 (T364822)
  • 14:11 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for tools-k8s-worker-nfs-9 (T364822)
  • 14:10 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9 (T364822)
  • 10:26 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:26 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-05-14

  • 13:28 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 13:28 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 07:48 dcaro: draining tools-k8s-worker-nfs-9 as it's stuck on IO
  • 07:48 dcaro@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=99) for node tools-k8s-worker-nfs-9
  • 07:48 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-9

2024-05-07

  • 16:23 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 16:23 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 12:21 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:21 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-05-06

  • 12:05 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 12:04 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 08:24 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 08:24 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 07:24 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 07:23 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api

2024-05-05

  • 07:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-nginx
  • 07:06 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-nginx

2024-05-03

  • 15:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 15:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 12:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 10:17 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:16 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-04-30

  • 10:56 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 10:55 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-04-26

  • 08:59 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 08:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 08:57 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 08:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-04-25

  • 12:57 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:57 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 09:48 taavi: update pywikibot script image to v9.1.0 T363132

2024-04-24

  • 15:30 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 15:29 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-04-18

  • 09:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 09:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-17

  • 20:49 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-50
  • 20:48 andrewbogott: In response to stuck processes (NFS?), running sudo cookbook wmcs.toolforge.k8s.reboot --hostname-list tools-k8s-worker-nfs-50 --cluster-name tools
  • 20:48 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-50
  • 15:21 dcaro: swapped login.toolforge.org to point to tools-bastion-13
  • 10:48 dcaro: rebooting tools-k8s-worker-nfs-1

2024-04-16

  • 11:08 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-1
  • 11:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-1
  • 08:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.apt.copy_to_main_repo (exit_code=0) for package 'python3-toolforge-weld' version '1.5.0'
  • 08:54 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.apt.copy_to_main_repo for package 'python3-toolforge-weld' version '1.5.0'

2024-04-15

  • 20:34 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 20:33 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 18:28 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 18:27 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 14:15 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:15 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 13:43 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 13:42 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 13:38 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 13:38 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 11:03 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 11:03 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 10:59 dcaro@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 10:59 dcaro@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 09:03 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 09:02 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-04-12

  • 10:14 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-admission
  • 10:14 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-admission
  • 09:35 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission
  • 09:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission
  • 09:27 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 09:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 01:19 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 01:18 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 01:18 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component calico
  • 01:17 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component ingress-admission
  • 01:17 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component calico
  • 01:17 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component api-gateway
  • 01:16 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component ingress-admission
  • 01:16 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component api-gateway
  • 01:15 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admission
  • 01:14 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admission
  • 01:13 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 01:12 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 01:11 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-04-11

  • 08:42 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 08:41 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-04-09

  • 17:21 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
  • 17:12 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
  • 17:11 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
  • 17:03 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
  • 16:57 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
  • 16:47 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
  • 14:23 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 14:23 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 14:23 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 14:22 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 14:22 wmbot~dcaro@urcuchillay: END (FAIL) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=99)
  • 14:22 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 14:11 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:11 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 13:43 dcaro: deployed builds-builder 0.0.94 and removed builds-admission
  • 13:39 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 13:38 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 12:21 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:21 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 12:19 dcaro: deploying toolforge-jobs-cli 16.0.6

2024-04-08

  • 16:35 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=0)
  • 16:24 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
  • 16:21 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
  • 16:11 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
  • 16:09 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_etcd_node (exit_code=99)
  • 16:09 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_etcd_node
  • 15:07 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
  • 14:49 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 14:49 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
  • 14:32 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 14:32 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=0)
  • 14:16 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 14:14 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-21
  • 14:13 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-21
  • 13:56 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:54 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:53 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-56
  • 13:53 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 13:52 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-56
  • 13:51 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:49 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:49 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:47 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:45 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:43 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:40 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:37 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:37 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:35 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 13:32 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 13:31 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 13:29 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 13:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 13:29 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 13:29 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:29 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 13:29 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 13:28 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 13:24 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:19 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 13:12 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_etcd_node (exit_code=99)
  • 13:12 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_etcd_node (T349207)
  • 10:26 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 10:26 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 08:55 dcaro_: deploy toolforge-jobs-framework-cli 16.0.5

2024-04-05

  • 12:15 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:15 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-04-03

  • 15:01 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 15:00 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:59 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:59 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:58 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:58 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:57 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:57 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:49 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:49 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:37 wmbot~raymond@ubuntu: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:37 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 11:24 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-proxy-06
  • 11:24 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance tools-proxy-06
  • 11:23 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-proxy-06
  • 11:23 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance tools-proxy-06
  • 11:21 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-proxy-06
  • 11:21 wmbot~taavi@runko: START - Cookbook wmcs.vps.remove_instance for instance tools-proxy-06
  • 09:45 taavi: rebuilding prebuild images for T361457

2024-04-02

  • 12:39 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-db-2 (T344717)
  • 12:38 fnegri@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-db-2 (T344717)
  • 07:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-docker-registry-05
  • 07:54 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-docker-registry-05

2024-03-28

  • 14:27 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-proxy-05
  • 14:26 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-proxy-05
  • 13:45 taavi: migrating toolforge.org floating IP from tools-proxy-06 to tools-proxy-7 T361223
  • 13:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-proxy'
  • 13:30 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-proxy'
  • 13:25 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-proxy'
  • 13:19 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-proxy'
  • 12:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-docker-registry-06
  • 12:12 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-docker-registry-06
  • 11:08 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-docker-registry'
  • 11:02 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-docker-registry'

2024-03-27

  • 12:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance toolserver-proxy-01
  • 12:19 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance toolserver-proxy-01

2024-03-26

  • 16:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-docker-registry-7.tools.eqiad1.wikimedia.cloud
  • 16:47 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-docker-registry-7.tools.eqiad1.wikimedia.cloud
  • 16:41 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-docker-registry-7.tools.eqiad1.wikimedia.cloud
  • 16:39 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-docker-registry-7.tools.eqiad1.wikimedia.cloud
  • 16:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-docker-registry'
  • 16:33 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-docker-registry'
  • 12:55 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-bastion-13.tools.eqiad1.wikimedia.cloud
  • 12:54 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-bastion-13.tools.eqiad1.wikimedia.cloud
  • 12:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-bastion'
  • 12:45 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-bastion'
  • 12:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-sgebastion-11
  • 12:43 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-sgebastion-11
  • 10:24 taavi: point toolserver.org DNS to tools-legacy-redirector-2 T311909

2024-03-25

  • 18:24 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-legacy-redirector
  • 18:23 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-legacy-redirector
  • 14:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-legacy-redirector-2.tools.eqiad1.wikimedia.cloud
  • 14:27 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-legacy-redirector-2.tools.eqiad1.wikimedia.cloud
  • 14:20 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-legacy-redirector-2.tools.eqiad1.wikimedia.cloud
  • 14:19 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-legacy-redirector-2.tools.eqiad1.wikimedia.cloud
  • 14:18 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-legacy-redirector-2.tools.eqiad1.wikimedia.cloud
  • 14:18 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-legacy-redirector-2.tools.eqiad1.wikimedia.cloud

2024-03-22

  • 11:43 dcaro: restarted sssd on tools-prometheus-6 as it was stopped (error)

2024-03-21

  • 15:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_haproxy_node (exit_code=0) for node tools-k8s-haproxy-4
  • 15:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_haproxy_node for node tools-k8s-haproxy-4
  • 15:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_haproxy_node (exit_code=0) for node tools-k8s-haproxy-3
  • 15:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_haproxy_node for node tools-k8s-haproxy-3
  • 15:42 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_haproxy_node (exit_code=99) for node toolsbeta-k8s-haproxy-3
  • 15:42 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_haproxy_node for node toolsbeta-k8s-haproxy-3
  • 15:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_haproxy_node (exit_code=0)
  • 15:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_haproxy_node
  • 12:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_haproxy_node (exit_code=0)
  • 12:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_haproxy_node

2024-03-20

  • 13:35 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-checker-04
  • 13:34 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-checker-04
  • 12:30 taavi: move checker service address to tools-checker-5
  • 11:24 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 11:24 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 10:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-checker-5.tools.eqiad1.wikimedia.cloud
  • 10:45 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-checker-5.tools.eqiad1.wikimedia.cloud
  • 10:40 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-checker-5.tools.eqiad1.wikimedia.cloud
  • 10:39 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-checker-5.tools.eqiad1.wikimedia.cloud
  • 10:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=0) with prefix 'tools-checker'
  • 10:34 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-checker'
  • 10:33 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'tools-checker'
  • 10:33 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-checker'
  • 10:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0)
  • 10:32 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase
  • 10:22 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.create_instance_with_prefix (exit_code=99) with prefix 'tools-checker'
  • 10:21 taavi@cloudcumin1001: START - Cookbook wmcs.vps.create_instance_with_prefix with prefix 'tools-checker'

2024-03-19

  • 21:28 taavi: kick off full container image rebuild for https://gerrit.wikimedia.org/r/1012753 (python3 backwards compat in lighttpd images) and https://gerrit.wikimedia.org/r/1010690 (add procps to base images)
  • 11:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-static-14
  • 11:21 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-static-14
  • 11:19 taavi: point dev.toolforge.org to tools-bastion-12 T314665
  • 10:26 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 10:25 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 09:38 dcaro: pushed docker-registry.tools.wmflabs.org/cloud-cicd-py311bookworm-tox:latest and docker-registry.tools.wmflabs.org/cloud-cicd-debian-builder-bookworm:2024-03-24.1 (T360405)

2024-03-18

  • 13:32 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-9 from 1.23.17 to 1.24.17 (T359638)
  • 13:31 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-9 from 1.23.17 to 1.24.17 (T359638)
  • 13:31 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-7 from 1.23.17 to 1.24.17 (T359638)
  • 13:30 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-7 from 1.23.17 to 1.24.17 (T359638)
  • 13:30 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-ingress-8 from 1.23.17 to 1.24.17 (T359638)
  • 13:29 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-ingress-8 from 1.23.17 to 1.24.17 (T359638)
  • 13:14 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-104 from 1.23.17 to 1.24.17 (T307651)
  • 13:13 taavi: restart harbor services after docker service restart
  • 13:13 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-104 from 1.23.17 to 1.24.17 (T307651)
  • 13:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-103 from 1.23.17 to 1.24.17 (T307651)
  • 13:12 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-103 from 1.23.17 to 1.24.17 (T307651)
  • 13:12 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-102 from 1.23.17 to 1.24.17 (T307651)
  • 13:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-102 from 1.23.17 to 1.24.17 (T307651)
  • 13:03 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-56 from 1.23.17 to 1.24.17 (T307651)
  • 13:02 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-56 from 1.23.17 to 1.24.17 (T307651)
  • 13:02 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-55 from 1.23.17 to 1.24.17 (T307651)
  • 13:01 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-55 from 1.23.17 to 1.24.17 (T307651)
  • 13:01 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-54 from 1.23.17 to 1.24.17 (T307651)
  • 13:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-54 from 1.23.17 to 1.24.17 (T307651)
  • 13:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-53 from 1.23.17 to 1.24.17 (T307651)
  • 12:59 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-53 from 1.23.17 to 1.24.17 (T307651)
  • 12:59 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-52 from 1.23.17 to 1.24.17 (T307651)
  • 12:58 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-52 from 1.23.17 to 1.24.17 (T307651)
  • 12:58 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-51 from 1.23.17 to 1.24.17 (T307651)
  • 12:57 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-51 from 1.23.17 to 1.24.17 (T307651)
  • 12:57 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-50 from 1.23.17 to 1.24.17 (T307651)
  • 12:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-50 from 1.23.17 to 1.24.17 (T307651)
  • 12:56 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-49 from 1.23.17 to 1.24.17 (T307651)
  • 12:55 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-49 from 1.23.17 to 1.24.17 (T307651)
  • 12:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-48 from 1.23.17 to 1.24.17 (T307651)
  • 12:53 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-48 from 1.23.17 to 1.24.17 (T307651)
  • 12:53 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-47 from 1.23.17 to 1.24.17 (T307651)
  • 12:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-47 from 1.23.17 to 1.24.17 (T307651)
  • 12:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-46 from 1.23.17 to 1.24.17 (T307651)
  • 12:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-46 from 1.23.17 to 1.24.17 (T307651)
  • 12:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-45 from 1.23.17 to 1.24.17 (T307651)
  • 12:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-45 from 1.23.17 to 1.24.17 (T307651)
  • 12:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-44 from 1.23.17 to 1.24.17 (T307651)
  • 12:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-44 from 1.23.17 to 1.24.17 (T307651)
  • 12:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-43 from 1.23.17 to 1.24.17 (T307651)
  • 12:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-43 from 1.23.17 to 1.24.17 (T307651)
  • 12:48 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-42 from 1.23.17 to 1.24.17 (T307651)
  • 12:47 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-42 from 1.23.17 to 1.24.17 (T307651)
  • 12:47 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-41 from 1.23.17 to 1.24.17 (T307651)
  • 12:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-41 from 1.23.17 to 1.24.17 (T307651)
  • 12:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-21 from 1.23.17 to 1.24.17 (T359638)
  • 12:44 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.23.17 to 1.24.17 (T359638)
  • 12:36 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-40 from 1.23.17 to 1.24.17 (T307651)
  • 12:35 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-40 from 1.23.17 to 1.24.17 (T307651)
  • 12:35 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-39 from 1.23.17 to 1.24.17 (T307651)
  • 12:34 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-39 from 1.23.17 to 1.24.17 (T307651)
  • 12:34 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-38 from 1.23.17 to 1.24.17 (T307651)
  • 12:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-filesystemtest-1
  • 12:33 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-38 from 1.23.17 to 1.24.17 (T307651)
  • 12:33 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-37 from 1.23.17 to 1.24.17 (T307651)
  • 12:32 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-filesystemtest-1
  • 12:32 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-37 from 1.23.17 to 1.24.17 (T307651)
  • 12:32 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-36 from 1.23.17 to 1.24.17 (T307651)
  • 12:31 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-36 from 1.23.17 to 1.24.17 (T307651)
  • 12:31 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-35 from 1.23.17 to 1.24.17 (T307651)
  • 12:30 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-35 from 1.23.17 to 1.24.17 (T307651)
  • 12:29 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-34 from 1.23.17 to 1.24.17 (T307651)
  • 12:28 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-34 from 1.23.17 to 1.24.17 (T307651)
  • 12:28 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-33 from 1.23.17 to 1.24.17 (T307651)
  • 12:27 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-33 from 1.23.17 to 1.24.17 (T307651)
  • 12:27 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-32 from 1.23.17 to 1.24.17 (T307651)
  • 12:26 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-32 from 1.23.17 to 1.24.17 (T307651)
  • 12:26 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-31 from 1.23.17 to 1.24.17 (T307651)
  • 12:25 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-31 from 1.23.17 to 1.24.17 (T307651)
  • 12:25 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-30 from 1.23.17 to 1.24.17 (T307651)
  • 12:24 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-30 from 1.23.17 to 1.24.17 (T307651)
  • 12:24 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-29 from 1.23.17 to 1.24.17 (T307651)
  • 12:23 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-29 from 1.23.17 to 1.24.17 (T307651)
  • 12:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-28 from 1.23.17 to 1.24.17 (T307651)
  • 12:22 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-28 from 1.23.17 to 1.24.17 (T307651)
  • 12:22 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-27 from 1.23.17 to 1.24.17 (T307651)
  • 12:21 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-27 from 1.23.17 to 1.24.17 (T307651)
  • 12:21 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-26 from 1.23.17 to 1.24.17 (T307651)
  • 12:20 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-26 from 1.23.17 to 1.24.17 (T307651)
  • 12:20 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-25 from 1.23.17 to 1.24.17 (T307651)
  • 12:19 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-25 from 1.23.17 to 1.24.17 (T307651)
  • 12:19 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-24 from 1.23.17 to 1.24.17 (T307651)
  • 12:18 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-24 from 1.23.17 to 1.24.17 (T307651)
  • 12:18 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-23 from 1.23.17 to 1.24.17 (T307651)
  • 12:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-acme-chief-4.tools.eqiad1.wikimedia.cloud
  • 12:15 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-23 from 1.23.17 to 1.24.17 (T307651)
  • 12:15 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-22 from 1.23.17 to 1.24.17 (T307651)
  • 12:14 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-acme-chief-4.tools.eqiad1.wikimedia.cloud
  • 12:11 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-22 from 1.23.17 to 1.24.17 (T307651)
  • 12:11 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-21 from 1.23.17 to 1.24.17 (T307651)
  • 12:05 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-21 from 1.23.17 to 1.24.17 (T307651)
  • 12:04 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-acme-chief-3.tools.eqiad1.wikimedia.cloud
  • 12:04 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-5 from 1.23.17 to 1.24.17 (T359638)
  • 12:03 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.23.17 to 1.24.17 (T359638)
  • 12:01 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-5 from 1.23.17 to 1.24.17 (T359638)
  • 12:01 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-acme-chief-3.tools.eqiad1.wikimedia.cloud
  • 12:00 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-acme-chief-3.tools.eqiad1.wikimedia.cloud
  • 12:00 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-acme-chief-3.tools.eqiad1.wikimedia.cloud
  • 11:56 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.23.17 to 1.24.17 (T359638)
  • 11:55 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-20 from 1.23.17 to 1.24.17 (T307651)
  • 11:54 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-20 from 1.23.17 to 1.24.17 (T307651)
  • 11:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-19 from 1.23.17 to 1.24.17 (T307651)
  • 11:53 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-19 from 1.23.17 to 1.24.17 (T307651)
  • 11:53 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-18 from 1.23.17 to 1.24.17 (T307651)
  • 11:52 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-18 from 1.23.17 to 1.24.17 (T307651)
  • 11:52 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-17 from 1.23.17 to 1.24.17 (T307651)
  • 11:51 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-17 from 1.23.17 to 1.24.17 (T307651)
  • 11:51 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-16 from 1.23.17 to 1.24.17 (T307651)
  • 11:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-16 from 1.23.17 to 1.24.17 (T307651)
  • 11:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-15 from 1.23.17 to 1.24.17 (T307651)
  • 11:49 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-15 from 1.23.17 to 1.24.17 (T307651)
  • 11:49 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-14 from 1.23.17 to 1.24.17 (T307651)
  • 11:48 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-14 from 1.23.17 to 1.24.17 (T307651)
  • 11:48 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-13 from 1.23.17 to 1.24.17 (T307651)
  • 11:47 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-13 from 1.23.17 to 1.24.17 (T307651)
  • 11:47 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-12 from 1.23.17 to 1.24.17 (T307651)
  • 11:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-12 from 1.23.17 to 1.24.17 (T307651)
  • 11:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-11 from 1.23.17 to 1.24.17 (T307651)
  • 11:45 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-11 from 1.23.17 to 1.24.17 (T307651)
  • 11:45 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-10 from 1.23.17 to 1.24.17 (T307651)
  • 11:43 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.23.17 to 1.24.17 (T307651)
  • 11:43 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-9 from 1.23.17 to 1.24.17 (T307651)
  • 11:42 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-9 from 1.23.17 to 1.24.17 (T307651)
  • 11:42 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-8 from 1.23.17 to 1.24.17 (T307651)
  • 11:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-8 from 1.23.17 to 1.24.17 (T307651)
  • 11:41 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-7 from 1.23.17 to 1.24.17 (T307651)
  • 11:40 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-7 from 1.23.17 to 1.24.17 (T307651)
  • 11:40 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-6 from 1.23.17 to 1.24.17 (T307651)
  • 11:39 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-6 from 1.23.17 to 1.24.17 (T307651)
  • 11:39 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-worker-nfs-5 from 1.23.17 to 1.24.17 (T307651)
  • 11:33 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-5 from 1.23.17 to 1.24.17 (T307651)
  • 11:33 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-4 from 1.23.17 to 1.24.17 (T307651)
  • 11:32 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-4 from 1.23.17 to 1.24.17 (T307651)
  • 11:32 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-3 from 1.23.17 to 1.24.17 (T307651)
  • 11:31 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-3 from 1.23.17 to 1.24.17 (T307651)
  • 11:31 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-2 from 1.23.17 to 1.24.17 (T307651)
  • 11:30 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-2 from 1.23.17 to 1.24.17 (T307651)
  • 11:30 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-1 from 1.23.17 to 1.24.17 (T307651)
  • 11:29 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-1 from 1.23.17 to 1.24.17 (T307651)
  • 11:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-9 from 1.23.17 to 1.24.17 (T359638)
  • 11:23 taavi: point tools-static proxy to tools-static-15 (bookworm) T311913
  • 11:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-static-15.tools.eqiad1.wikimedia.cloud
  • 11:17 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-static-15.tools.eqiad1.wikimedia.cloud
  • 11:17 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-static-15.tools.eqiad1.wikimedia.cloud
  • 11:17 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-static-15.tools.eqiad1.wikimedia.cloud
  • 11:17 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-9 from 1.23.17 to 1.24.17 (T359638)
  • 11:13 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-8 from 1.23.17 to 1.24.17 (T359638)
  • 11:08 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-8 from 1.23.17 to 1.24.17 (T359638)
  • 11:01 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 11:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 11:00 aborrero@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=97) for component jobs-api
  • 11:00 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 11:00 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-7 from 1.23.17 to 1.24.17 (T359638)
  • 10:53 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.23.17 to 1.24.17 (T359638)
  • 10:53 aborrero@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=99) for node tools-k8s-7 from 1.23.17 to 1.24.17 (T359638)
  • 10:53 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-7 from 1.23.17 to 1.24.17 (T359638)
  • 10:47 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.prepare_upgrade (exit_code=0) for cluster tools upgrade from 1.23.17 to 1.24.17 (T307651)
  • 10:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.prepare_upgrade for cluster tools upgrade from 1.23.17 to 1.24.17 (T307651)
  • 10:04 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-bastion-12.tools.eqiad1.wikimedia.cloud
  • 10:03 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-bastion-12.tools.eqiad1.wikimedia.cloud
  • 09:27 taavi: deleted shutdown grid engine VMs T314664

2024-03-15

  • 10:50 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 10:50 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-03-14

  • 17:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.apt.copy_to_main_repo (exit_code=0) for package 'misctools' version '1.48'
  • 17:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.apt.copy_to_main_repo for package 'misctools' version '1.48'
  • 15:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-docker-imagebuilder-01
  • 15:16 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-docker-imagebuilder-01
  • 15:11 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.remove_instance (exit_code=99) for instance tools-docker-imagebuilder-01
  • 15:11 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-docker-imagebuilder-01
  • 15:10 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.remove_instance (exit_code=99) for instance tools-docker-imagebuilder-01
  • 15:09 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-docker-imagebuilder-01
  • 11:02 taavi: stop grid related VMs T314664
  • 11:01 taavi: disable grid access for remaining tools still running on the grid T314664

2024-03-13

  • 19:21 andrewbogott: shutting down old puppet infra: tools-puppetmaster-02 and tools-puppetdb-1. These can be deleted in a week or two presuming everything remains stable.

2024-03-12

  • 12:38 taavi: hard reboot tools-prometheus-6
  • 11:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 11:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-03-11

  • 16:46 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 16:46 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 13:20 arturo: cached registry.k8s.io/kube-state-metrics/kube-state-metrics:v2.6.0 as docker-registry.tools.wmflabs.org/kube-state-metrics:v2.6.0 in the docker registry for T359798

2024-03-09

  • 12:48 taavi: hard reboot tools-sgebastion-10 due to stuck NFS procs

2024-03-08

  • 12:02 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 12:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-03-07

  • 14:33 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 13:42 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 13:41 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-03-06

  • 10:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-32
  • 10:47 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_grid_node (exit_code=1) for tools-sgeweblight-10-17, tools-sgeweblight-10-32
  • 10:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-17, tools-sgeweblight-10-32
  • 10:34 taavi: rebuilding all docker images for https://gerrit.wikimedia.org/r/c/operations/docker-images/toollabs-images/+/1005952 (T293552) + normal package updates
  • 09:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
  • 09:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors
  • 09:42 taavi: reboot tools-sgeexec-10-20, -21, -23, sgeweblight-10-32 due to stuck nfs procs

2024-03-05

  • 16:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-imagebuilder-2.tools.eqiad1.wikimedia.cloud
  • 16:11 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-imagebuilder-2.tools.eqiad1.wikimedia.cloud
  • 16:09 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 16:09 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 16:07 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0)
  • 16:07 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase
  • 16:06 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.openstack.quota_increase (exit_code=97) (T357901)
  • 16:06 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase (T357901)
  • 16:05 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=99) on tools-imagebuilder-2.tools.eqiad1.wikimedia.cloud
  • 16:04 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-imagebuilder-2.tools.eqiad1.wikimedia.cloud

2024-03-04

  • 17:56 bd808@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 17:56 bd808@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 16:57 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 16:57 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:43 taavi: reboot tools-sgegrid-shadow due to high number of procs in D state

2024-03-03

  • 10:38 dcaro: reboot tools-k8s-worker-nfs-55 got nfs lockup (logrotate in D state)

2024-03-01

  • 21:14 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 21:14 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api

2024-02-29

  • 14:36 dcaro: deploy webservice 0.103.3

2024-02-28

  • 11:57 dcaro: deploy tools-webservice 0.103.2 with probes (T341919)
  • 00:46 bd808@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 00:46 bd808@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-02-26

  • 09:54 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) (T284656)
  • 09:54 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node (T284656)
  • 09:35 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the tools cluster
  • 09:35 aborrero@cloudcumin1001: Added a new k8s control tools-k8s-control-9.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:26 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the tools cluster (T284656)

2024-02-23

  • 14:19 taavi: remove isc-dhcp-server (server, not client) from tools-db-2
  • 13:32 taavi: remove toolschecker alerts for grid engine jobs T358333

2024-02-22

  • 14:26 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 14:26 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 14:24 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:24 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:17 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-api
  • 14:17 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 14:07 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component envvars-api
  • 14:07 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 14:03 sstefanova@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component envvars-api
  • 14:03 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 11:23 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) (T284656)
  • 11:23 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node (T284656)
  • 11:15 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 11:15 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-104.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:06 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 10:52 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 10:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 09:39 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the tools cluster
  • 09:39 aborrero@cloudcumin1001: Added a new k8s control tools-k8s-control-8.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:29 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the tools cluster (T284656)
  • 08:04 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-51
  • 08:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-51
  • 08:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-38
  • 08:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-38
  • 08:02 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-nfs-25
  • 08:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-nfs-25

2024-02-21

  • 17:07 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 17:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 15:48 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component envvars-api
  • 15:48 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component envvars-api
  • 14:41 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:40 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 14:34 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 14:21 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 14:20 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 09:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-control-4
  • 09:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-control-4
  • 09:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a control role in the tools cluster
  • 09:20 taavi@cloudcumin1001: Added a new k8s control tools-k8s-control-7.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:10 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a control role in the tools cluster

2024-02-20

  • 16:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 16:12 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-103.tools.eqiad1.wikimedia.cloud to the cluster
  • 16:05 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.worker.drain (exit_code=0) for node tools-k8s-worker-102
  • 16:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.worker.drain for node tools-k8s-worker-102
  • 16:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 15:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-101
  • 15:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-101
  • 15:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 15:48 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-102.tools.eqiad1.wikimedia.cloud to the cluster
  • 15:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 15:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-102
  • 15:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-102
  • 15:38 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker role in the tools cluster
  • 15:38 taavi@cloudcumin1001: Added a new k8s worker tools-k8s-worker-102.tools.eqiad1.wikimedia.cloud to the cluster
  • 15:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 15:23 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-k8s-worker-nfs-51.tools.eqiad1.wikimedia.cloud
  • 15:21 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-k8s-worker-nfs-51.tools.eqiad1.wikimedia.cloud
  • 12:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:57 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-56.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-100
  • 12:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-100
  • 12:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:40 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-55.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-99
  • 12:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-99
  • 12:29 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:29 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-54.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:20 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:19 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-98
  • 12:19 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-98
  • 12:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:18 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-53.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-97
  • 12:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-97
  • 11:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:56 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-52.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:45 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-96
  • 11:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-96
  • 11:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:36 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-51.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:26 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-50.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:16 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-49.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:05 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-95
  • 11:04 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-95
  • 10:58 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-94
  • 10:57 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-94
  • 10:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-93
  • 10:56 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-93
  • 10:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 10:56 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-48.tools.eqiad1.wikimedia.cloud to the cluster
  • 10:45 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 10:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-92
  • 10:44 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-92
  • 09:53 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-ingress-6
  • 09:52 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-ingress-6
  • 09:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the tools cluster
  • 09:46 taavi@cloudcumin1001: Added a new k8s ingress tools-k8s-ingress-9.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:41 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-47.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the tools cluster
  • 09:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-91
  • 09:29 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-91
  • 09:15 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:15 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-46.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:02 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 09:00 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-90
  • 08:59 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-90
  • 08:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:57 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-45.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-89
  • 08:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-89
  • 08:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:47 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-44.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:38 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-88
  • 08:36 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-88

2024-02-19

  • 19:04 wmbot~raymond@ubuntu: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 19:03 wmbot~raymond@ubuntu: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 13:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-ingress-5
  • 13:16 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-ingress-5
  • 13:09 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 13:09 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-43.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:59 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:58 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-87
  • 12:58 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-87
  • 12:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:56 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-42.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-86
  • 12:44 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-86
  • 12:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:44 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-41.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.openstack.quota_increase (exit_code=0) (T357901)
  • 12:33 taavi@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase (T357901)
  • 12:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-k8s-worker-nfs-38.tools.eqiad1.wikimedia.cloud
  • 12:32 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-k8s-worker-nfs-38.tools.eqiad1.wikimedia.cloud
  • 12:24 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 12:23 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:20 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-85
  • 12:19 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-85
  • 12:18 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:18 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-40.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:08 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-84
  • 12:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-84
  • 12:04 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:04 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-39.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:54 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:53 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-83
  • 11:53 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-83
  • 11:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:50 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-38.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:40 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:40 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-82
  • 11:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-82
  • 11:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:39 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-37.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:28 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:28 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-81
  • 11:27 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-81
  • 09:03 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 09:03 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 08:57 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 08:57 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-02-16

  • 15:28 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 15:27 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 12:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the tools cluster
  • 12:21 taavi@cloudcumin1001: Added a new k8s ingress tools-k8s-ingress-8.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:14 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the tools cluster
  • 10:37 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 10:32 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 10:32 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 10:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 10:31 wmbot~dcaro@urcuchillay: END (ERROR) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=255)
  • 10:31 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 09:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:59 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-36.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:49 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:49 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-80
  • 09:49 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-80
  • 09:45 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:45 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-35.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:35 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-79
  • 09:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-79
  • 09:24 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:24 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-34.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:13 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:06 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-78
  • 09:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-78
  • 09:05 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:05 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-33.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:55 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:55 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-77
  • 08:54 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-77

2024-02-15

  • 13:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-ingress-4
  • 13:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-ingress-4
  • 13:02 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 13:02 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-32.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-76
  • 12:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-76
  • 12:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:44 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-31.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-75
  • 12:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-75
  • 11:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a ingress role in the tools cluster
  • 11:37 taavi@cloudcumin1001: Added a new k8s ingress tools-k8s-ingress-7.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the tools cluster
  • 11:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-k8s-ingress-7
  • 11:29 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-k8s-ingress-7
  • 11:29 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a ingress role in the tools cluster
  • 11:24 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a ingress role in the tools cluster

2024-02-14

  • 19:32 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-17, tools-sgeweblight-10-30
  • 16:35 taavi: kill jobs user 'wikishizhao' is running directly on the grid per https://wikitech.wikimedia.org/wiki/Help:Toolforge/Rules #3
  • 16:30 taavi: reboot tools-sgeexec-10-23 due to high load
  • 09:14 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-k8s-worker-nfs-25.tools.eqiad1.wikimedia.cloud
  • 09:13 taavi@cloudcumin1001: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-k8s-worker-nfs-25.tools.eqiad1.wikimedia.cloud
  • 09:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.openstack.cloudvirt.vm_console (exit_code=0)
  • 09:07 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.openstack.cloudvirt.vm_console
  • 09:07 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:07 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-30.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:56 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-74
  • 08:55 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-74
  • 08:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:54 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-29.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:44 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:44 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-73
  • 08:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-73
  • 08:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:43 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-28.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-72
  • 08:32 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-72
  • 08:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:32 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-27.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:23 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-71
  • 08:22 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-71
  • 08:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:21 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-26.tools.eqiad1.wikimedia.cloud to the cluster
  • 08:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:08 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-70
  • 08:07 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-70
  • 08:05 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 08:05 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-25.tools.eqiad1.wikimedia.cloud to the cluster
  • 07:56 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 07:54 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-69
  • 07:54 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-69
  • 07:53 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 07:53 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-24.tools.eqiad1.wikimedia.cloud to the cluster
  • 07:44 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 07:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-68
  • 07:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-68

2024-02-13

  • 15:42 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-67
  • 15:41 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-67
  • 15:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 15:41 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-23.tools.eqiad1.wikimedia.cloud to the cluster
  • 15:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-66
  • 15:30 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-66
  • 15:30 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 15:30 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-22.tools.eqiad1.wikimedia.cloud to the cluster
  • 15:19 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 15:17 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-65
  • 15:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-65
  • 09:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:36 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-21.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-64
  • 09:25 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-64

2024-02-12

  • 14:58 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 14:58 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-20.tools.eqiad1.wikimedia.cloud to the cluster
  • 14:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 14:48 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-62
  • 14:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-62
  • 14:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 14:47 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-19.tools.eqiad1.wikimedia.cloud to the cluster
  • 14:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 14:26 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-61
  • 14:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-61
  • 13:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-60
  • 13:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-60
  • 13:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 13:43 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-18.tools.eqiad1.wikimedia.cloud to the cluster
  • 13:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 13:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-59
  • 13:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-59
  • 13:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-58
  • 13:32 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-58
  • 13:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 13:22 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-17.tools.eqiad1.wikimedia.cloud to the cluster
  • 13:12 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 13:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-57
  • 13:10 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-57
  • 13:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-56
  • 13:09 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-56
  • 13:09 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 13:09 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-16.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:59 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:59 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-55
  • 12:58 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-55
  • 12:58 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-54
  • 12:57 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-54
  • 12:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:56 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-15.tools.eqiad1.wikimedia.cloud to the cluster
  • 12:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-k8s-worker-nfs-15
  • 12:45 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-k8s-worker-nfs-15
  • 12:44 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 12:37 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 12:37 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-53
  • 12:36 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-53
  • 12:36 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-52
  • 12:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-52
  • 10:51 sstefanova@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api
  • 10:50 sstefanova@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api
  • 10:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 10:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers

2024-02-11

  • 11:39 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
  • 11:39 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2024-02-09

  • 18:03 andrewbogott: updated the default security group, removing the 0.0.0.0/0 rule allowing port 22 access everywhere, replaced it with a 172.16.0.0/21 rule
  • 13:06 taavi: reboot tools-sgecron-2 due to high load
  • 10:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component image-config
  • 10:34 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component image-config
  • 09:56 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:56 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-14.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:47 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:47 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-51
  • 09:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-51
  • 09:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-50
  • 09:46 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-50
  • 08:56 dcaro: restart tools-k8s-worker-50 due to D some stuck processes

2024-02-08

  • 13:03 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
  • 13:03 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors
  • 09:46 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:46 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-13.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:35 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:34 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-49
  • 09:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-49
  • 09:33 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-48
  • 09:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-48
  • 09:32 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:32 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-12.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:23 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-47
  • 09:22 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-47
  • 09:22 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-46
  • 09:21 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-46
  • 09:21 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:21 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-11.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:13 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 09:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-45
  • 09:11 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-45
  • 09:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-44
  • 09:10 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-44
  • 09:10 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 09:10 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-10.tools.eqiad1.wikimedia.cloud to the cluster
  • 09:00 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:59 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 08:58 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 08:58 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-43
  • 08:57 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-43
  • 08:57 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-42
  • 08:56 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-42

2024-02-07

  • 21:33 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all workers
  • 18:00 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all workers
  • 17:58 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-9
  • 17:58 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-9
  • 17:24 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for all workers
  • 17:23 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all workers
  • 17:05 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for all workers
  • 17:05 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all workers
  • 17:03 taavi@cloudcumin1001: END (ERROR) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=97) for all workers
  • 17:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all workers
  • 17:01 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=99) for all workers
  • 16:04 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all workers

2024-02-06

  • 13:09 aborrero@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for all nodes (T356507)
  • 11:50 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 11:50 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 11:16 aborrero@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.reboot for all nodes (T356507)

2024-01-31

  • 14:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 14:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-01-30

  • 19:24 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 19:24 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-9.tools.eqiad1.wikimedia.cloud to the cluster
  • 19:17 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 19:16 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-k8s-worker-nfs-9
  • 19:16 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-k8s-worker-nfs-9
  • 19:16 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 19:13 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 19:12 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 19:12 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-8.tools.eqiad1.wikimedia.cloud to the cluster
  • 19:04 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 19:04 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-k8s-worker-nfs-8
  • 19:03 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-k8s-worker-nfs-8
  • 18:51 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 18:48 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 18:48 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.vps.remove_instance (exit_code=0) for instance tools-k8s-worker-nfs-8
  • 18:47 taavi@cloudcumin1001: START - Cookbook wmcs.vps.remove_instance for instance tools-k8s-worker-nfs-8
  • 18:46 taavi@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 18:42 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 18:41 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 18:41 taavi@cloudcumin1001: Added a new k8s worker-nfs tools-k8s-worker-nfs-7.tools.eqiad1.wikimedia.cloud to the cluster
  • 18:33 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 18:29 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-41
  • 18:29 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-41
  • 18:24 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-40
  • 18:23 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-40
  • 18:22 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-39
  • 18:22 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-39
  • 18:18 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-38
  • 18:17 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-38
  • 18:09 andrew@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-37
  • 18:08 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-37
  • 15:16 dcaro: restart harbor now that the db is clean (T356037)
  • 15:14 dcaro: restart harbor now that the db is clean (T3543)
  • 13:08 taavi: create no-op DMARC record T354112
  • 12:39 dcaro: rebuilding all the toolforge images (T354320)
  • 10:16 dcaro: restarting harbor and flushing redis to regenerate cache data (T356037)
  • 09:33 dcaro: cleaning up old schedules on harbor (T356037)

2024-01-29

  • 19:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-36
  • 19:46 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=99) for host tools-k8s-worker-36
  • 19:46 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-36
  • 14:36 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.vps.refresh_puppet_certs (exit_code=0) on tools-mail-4.tools.eqiad1.wikimedia.cloud
  • 14:34 wmbot~taavi@runko: START - Cookbook wmcs.vps.refresh_puppet_certs on tools-mail-4.tools.eqiad1.wikimedia.cloud
  • 12:06 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 12:06 wmbot~taavi@runko: Added a new k8s worker-nfs tools-k8s-worker-nfs-6.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:55 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:51 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 11:51 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:37 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:37 wmbot~taavi@runko: Added a new k8s worker-nfs tools-k8s-worker-nfs-5.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:26 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:23 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 11:22 wmbot~taavi@runko: Added a new k8s worker-nfs tools-k8s-worker-nfs-4.tools.eqiad1.wikimedia.cloud to the cluster
  • 11:12 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:12 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-35
  • 11:10 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-35
  • 11:10 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-34
  • 11:09 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-34
  • 11:09 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-33
  • 11:07 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-33
  • 11:06 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.remove_k8s_node (exit_code=0) for host tools-k8s-worker-32
  • 11:04 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-32
  • 11:01 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-31
  • 10:59 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.remove_k8s_node for host tools-k8s-worker-30
  • 10:57 wmbot~taavi@runko: END (FAIL) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=99) for a worker-nfs role in the tools cluster
  • 10:56 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 10:51 wmbot~taavi@runko: END (PASS) - Cookbook wmcs.toolforge.add_k8s_node (exit_code=0) for a worker-nfs role in the tools cluster
  • 10:51 wmbot~taavi@runko: Added a new k8s worker-nfs tools-k8s-worker-nfs-3.tools.eqiad1.wikimedia.cloud to the cluster
  • 10:46 blancadesal: increased harbor quota for wd-shex-infer to 2GiB
  • 10:44 blancadesal: increased harbor quota for lucaswerkmeister-test to 2GiB
  • 10:31 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 10:31 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
  • 10:31 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2024-01-26

  • 10:56 taavi: copy helmfile_0.144.0-1_all to bookworm-tools, bookworm-toolsbeta

2024-01-25

  • 13:17 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 13:04 wmbot~taavi@runko: START - Cookbook wmcs.toolforge.add_k8s_node for a worker-nfs role in the tools cluster
  • 11:13 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 11:12 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-24

  • 09:54 dcaro: deploy toolforge-jobs-framework-cli 16.0.1

2024-01-23

  • 19:11 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 19:11 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 14:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 14:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 14:43 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component wmcs-k8s-metrics
  • 14:43 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component wmcs-k8s-metrics
  • 13:31 taavi: rebooting tools-sgeexec-10-21, tools-sgeexec-10-22
  • 12:58 dcaro: deployed toolforge-envvars-cli 0.0.4
  • 10:23 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 10:23 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-19

  • 15:40 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 15:40 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 12:11 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 12:10 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api

2024-01-18

  • 12:24 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.add_k8s_node for a worker role in the tools cluster
  • 12:21 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeexec-10-17

2024-01-17

  • 18:16 dhinus: increase volume quotas for toolsdb T344717
  • 18:14 fnegri@cloudcumin1001: END (FAIL) - Cookbook wmcs.openstack.quota_increase (exit_code=99) (T344717)
  • 18:14 fnegri@cloudcumin1001: START - Cookbook wmcs.openstack.quota_increase (T344717)
  • 14:34 wmbot~dcaro@urcuchillay: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 14:34 wmbot~dcaro@urcuchillay: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 08:56 taavi: update all pre-built docker images T352886

2024-01-15

  • 09:18 taavi: reboot stuck tools-k8s-worker-84

2024-01-12

  • 09:07 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.apt.copy_to_main_repo (exit_code=0) for package 'toolforge-builds-cli' version '0.0.12'
  • 09:07 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.apt.copy_to_main_repo for package 'toolforge-builds-cli' version '0.0.12'

2024-01-11

  • 17:30 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 17:12 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 17:12 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 15:14 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 15:13 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder

2024-01-10

  • 22:02 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component maintain-kubeusers
  • 22:02 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component maintain-kubeusers
  • 09:17 taavi: reboot tools-k8s-worker-98

2024-01-09

  • 23:37 andrewbogott: restarting harbor-db in an attempt to reform harbor -- T354714
  • 23:30 andrewbogott: rebooting tools-harbor-1 in a feeble attempt to get it to work (docker-compose can't restart it)
  • 23:12 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds-builder
  • 23:12 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 23:11 andrew@cloudcumin1001: END (FAIL) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=99) for component builds.builder
  • 23:11 andrew@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds.builder
  • 17:31 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 17:30 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 10:13 taavi: reboot tools-sgeexec-10-17 due to high load

2024-01-08

  • 12:26 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.remove_grid_node for tools-sgeweblight-10-27, tools-sgeweblight-10-28
  • 10:51 taavi@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component jobs-api
  • 10:51 taavi@cloudcumin1001: START - Cookbook wmcs.toolforge.k8s.component.deploy for component jobs-api
  • 10:17 taavi: reboot tools-sgeexec-10-21

2024-01-05

  • 14:55 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 14:55 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 11:56 wm-bot2: dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-builder
  • 11:55 wm-bot2: dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-builder
  • 10:29 fnegri@cloudcumin1001: END (PASS) - Cookbook wmcs.toolforge.grid.cleanup_queue_errors (exit_code=0)
  • 10:29 fnegri@cloudcumin1001: START - Cookbook wmcs.toolforge.grid.cleanup_queue_errors

2024-01-04

  • 10:11 dcaro: deploy toolforge-envvars-cli 0.0.3

2024-01-03

  • 21:22 andrewbogott: truncating 200 logfiles to 5M on tools nfs
  • 21:17 andrewbogott: deleting many stray core dumps throughout nfs storage

2024-01-02

  • 11:06 dcaro: restart toolsdb database to flush connections (T354176)
  • 10:42 dcaro: flushed the redis db on tools-harbor-1 (T354176)
  • 10:37 dcaro: hard reboot tools-harbor-1
  • 10:13 dhinus: hard reboot tools-harbor-1

2024-01-01

  • 15:55 andrewbogott: rebooting tools-harbor-1, T354151

Archives

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Tools/SAL&oldid=2254455"