08:57 arturo: restart shinken service. Was in failure state, and is working after that
08:41 arturo: add myself as projectadmin to investigate shinken issues
2019-08-09
10:53 arturo: rebalance load: reallocating shinken-02 from cloudvirt1018 to cloudvirt1006
2019-07-25
16:12 jeh: restarting shinken after NFS maintenance T224228
2019-06-03
15:21 andrewbogott: stopping shinken and puppet on shinken-02 because there are about to be a lot of alerts related to the rebuild of cloudservices1003
2019-04-11
19:55 andrewbogott: re-enabling puppet and the shinken service
19:35 andrewbogott: disabling shinken service and puppet on shinken-02 to keep things quiet while the tools puppetmaster moves
2019-03-27
14:43 gtirloni: deployed shinken-puppetmaster-01
2019-03-11
18:09 gtirloni: restarted shinken to clear invalid alerts
2019-02-22
13:36 gtirloni: rebooted shinken-02
13:18 gtirloni: re-enabled puppet on shinken-02
2019-02-06
13:57 gtirloni: rebooted shinken-02
2018-11-26
13:36 gtirloni: re-enabled puppet
2018-11-19
18:26 andrewbogott: stopping puppet and shinken service on shinken-02 during the deployment-prep migration
2018-11-16
12:50 gtirloni: started shinken service
11:42 gtirloni: stopped shinken service temporarily
2018-11-13
19:21 andrewbogott: moving shinken-02 to eqiad1-r
2018-11-06
14:02 gtirloni: re-enabled shinken and puppet
2018-10-29
20:25 Krenair: made /var/log/ircecho/* files world-readable so ircecho (running as 'nobody') can read them for output to IRC (they were written by the shinken user)
02:11 Krenair: After much fiddling around, shinken-wm should be working properly again. Puppet is disabled on the host until https://gerrit.wikimedia.org/r/455277 goes in
2018-08-24
20:32 Krenair: more details about IRC bot problems in #wikimedia-releng
19:58 Krenair: Restarting ircecho on shinken-01, messages have been going into /var/log/ircecho/irc-releng.log but not arriving in IRC
2018-06-07
20:04 Krenair: Enabling puppet and running on shinken-01. "The last Puppet run was at Wed Jun 6 15:58:12 UTC 2018 (1653 minutes ago). Puppet is disabled. silencing shinken during labvirt reboots"
2018-06-06
14:05 andrewbogott: stopping shinken for now; too noisy
2018-02-22
21:15 madhuvishy: add Chico Venancio as member and project admin
2018-02-09
01:09 bd808: Removed Yuvipanda at user request (T186289)
2018-01-16
20:02 madhuvishy: reboot shinken-01
04:11 bd808: Restarted ircecho service on shinken-01.shinken.eqiad.wmflabs
2018-01-04
23:21 Krenair: Noticed shinken-wm was missing from IRC, restarted ircecho service
2017-12-11
18:12 andrewbogott: re-enabling shinken and puppet on shinken-01
14:42 andrewbogott: stopping the shinken service on shinken-01 and disabling puppet so it stays off for a while
2017-08-03
19:27 madhuvishy: Upgrade python-novaclient on shinken-01
2017-06-14
22:15 madhuvishy: Stopped service ircecho
2017-06-01
21:29 bd808: Restarted ircecho
21:28 bd808: Forced a puppet run to pick up changes for T166420
2017-05-04
16:50 madhuvishy: restarted ircecho and reenabled puppet
16:05 madhuvishy: Stopped ircecho for grid master backup
2017-01-19
07:22 madhuvishy: Bringing back shinken post T154336
2017-01-18
17:09 madhuvishy: Silencing shinken for nfs misc migration T154336
2016-11-18
16:23 Krenair: re-enabling and running puppet
2016-11-14
16:18 madhuvishy: Stopped irc-echo and puppet on shinken-01 for T146154
2016-11-01
22:54 Krenair: enabled puppet, shinken is back
2016-10-05
13:08 andrewbogott: removing labs_debrepo from puppet for shinken-01 because the repo is empty and seemingly unused.
2016-01-28
15:27 scfc_de: shinken-01: Rebooted to get a baseline for T123271.
2015-09-09
19:10 valhallasw`cloud: disabling irc again
18:33 valhallasw`cloud: alerts have subsided, restarting ircecho
16:53 valhallasw`cloud: so it's sudo /etc/init.d/ircecho stop. Restarting init.d/shinken.
16:48 valhallasw`cloud: eh, which shinken process should I kill to silence shinken-wm_ ? did /etc/init.d/shinken stop now
2015-08-25
09:40 YuviPanda: restarted ircecho on shinken-01
2015-08-10
16:03 valhallasw`cloud: scheduler, poller, reactionner, broker, receiver and arbiter were all down. service shinken start'ed
February 2
12:52 yuvipanda: started shinken-server-01 manually from virt1000
December 25
00:52 YuviPanda: built and set up python3-ldap package for shinkengen
December 11
07:45 YuviPanda: shut up shinken-wm
November 26
10:38 YuviPanda: killed old instances shinken-server and tustyshinkencheck
November 25
11:24 YuviPanda: made /usr/bin/mail a small script that just logs to /data/project/shinken-server-01.email on shinken-server-01
11:06 YuviPanda: ln -s /bin/true /usr/bin/mail on shinken-server-01 to prevent it from spamming people same time shinken-01 does. See https://phabricator.wikimedia.org/T75859
September 15
13:14 YuviPanda: added yuvipanda to project to mess around
