Jump to content

Nova Resource:Tools.heritage/SAL

From Wikitech

2024-11-19

  • 17:54 wmbot~multichill@tools-bastion-12: loaded the jobs.yml, job list was empty, and started update-monuments

2024-09-01

2024-08-22

  • 05:37 wmbot~jeanfred@tools-sgebastion-10: Started harvesting again with 4GB memory

2024-08-19

2023-10-15

2023-08-31

  • 22:01 wm-bot: <jeanfred> Enable check_emailable_users crontab for WLM 2023

2023-08-26

2023-08-23

  • 07:13 wm-bot: <jeanfred> Removed STRICT_TRANS_TABLES from SQL_MODE (again) for T338987

2023-08-22

2023-08-21

2023-05-16

  • 12:22 wm-bot: <jeanfred> Run check_emailable_users.py -category:"Images_from_Wiki_Loves_Earth_2023" -delta:23040 -notify to catch up with the first 16 days of WLE 2023

2023-03-08

2022-10-03

  • 09:55 wm-bot: <lokal-profil> Deploy 8b52299

2022-09-08

  • 19:20 wm-bot: <jeanfred> Run check_emailable_users.py -category:"Images_from_Wiki_Loves_Monuments_2022" -delta:13000 -notify to catch up with the first 8 days of WLM 2022
  • 18:53 wm-bot: <jeanfred> Deploy ae766f9

2022-09-07

  • 20:18 wm-bot: <jeanfred> Run check_emailable_users.py -category:"Images_from_Wiki_Loves_Monuments_2022" -delta:10500 -notify to catch up with the first 7 days of WLM 2022

2022-08-31

2022-08-29

2022-05-03

  • 08:18 wm-bot: <jeanfred> Trigger a full update_monuments job post-T307269
  • 08:17 wm-bot: <jeanfred> Run ./bin/build.sh to rebuild the virtualenv with Python3.7 on a Buster node (for T307269)
  • 08:16 wm-bot: <jeanfred> Run ./bin/build.sh to rebuild the virtualenv with Python3.7 on a Buster node
  • 08:07 wm-bot: <jeanfred> Deploy latest from Git: ef4111a, f70063a, db03f84, 27fa2f6

2022-04-27

  • 21:13 wm-bot: <jeanfred> Manual trigger of categorize_images.sh

2022-04-18

  • 19:56 wm-bot: <lokal-profil> Adding the -release buster flag to cronjobs as part of Stretch deprecation

2022-03-03

2022-02-02

  • 12:49 wm-bot: <jeanfred> Trigger full update_monuments.sh run now that T300252 is fixed
  • 12:47 wm-bot: <jeanfred> Deploy 2715e85 (T300252)
  • 12:30 urbanecm: Run P19935 in `s51138__heritage_p` to workaround T300252 for Czech
  • 11:42 urbanecm: Run `python erfgoedbot/update_database.py -countrycode:cz -langcode:cs` with 759229 applied to ensure fix for T300252 works

2022-01-27

2021-09-06

  • 11:21 wm-bot: <jeanfred> Run check_emailable_users.py -category:"Images_from_Wiki_Loves_Monuments_2021" -delta:10500 -notify to catch up with the first 7 days of WLM 2021
  • 11:19 wm-bot: <jeanfred> Bumped check_emailable_users to 2021 and re-enabled cron job

2021-09-04

2021-08-26

2020-09-04

  • 21:22 wm-bot: <lokal-profil> Deploy latest from Git master: 0f24d39

2020-09-03

2020-09-02

  • 08:00 wm-bot: <lokal-profil> Deploy latest from Git master: 037df2f, 9c73470

2020-08-31

  • 22:02 wm-bot: <lokal-profil> Manually triggered update_monuments (after temporarily setting setuptools==49.3.0)
  • 13:55 wm-bot: <lokal-profil> Manually triggered update_monuments (again, again)
  • 13:53 wm-bot: <lokal-profil> Deploy latest from Git master: 069df77, 907c6fe (T224405), 21d30e0, 25a1951

2020-08-30

  • 21:10 wm-bot: <lokal-profil> Manually triggered update_monuments (again)
  • 21:06 wm-bot: <lokal-profil> Forced re-install of pywikibot over pip
  • 20:40 wm-bot: <lokal-profil> Bumped check_emailable_users to 2020 and re-enabled cron job
  • 20:36 wm-bot: <lokal-profil> Manually triggered update_monuments
  • 20:19 wm-bot: <lokal-profil> rename existing .venv and re-run build.sh
  • 11:32 wm-bot: <lokal-profil> Deploy latest from Git master: 0955c33 (T224405)

2020-08-28

  • 08:27 wm-bot: <lokal-profil> Deploy latest from Git master: 415c004, f3ac064

2020-08-26

  • 21:29 wm-bot: <jeanfred> jsub new categorize_images job to test 7b0a9d7
  • 21:29 wm-bot: <jeanfred> Deploy latest from Git master: 7b0a9d7

2020-08-25

2020-07-06

  • 22:55 wm-bot: <root> Migrated .webservicerc to service.template (T257229)

2020-02-25

  • 23:17 wm-bot: <root> Migrated to 2020 Kubernetes cluster

2020-02-24

2020-02-11

  • 20:37 wm-bot157: <lokal-profil> Triggered a manual categorisation job for ua_uk (T244333)

2020-02-05

  • 08:50 wm-bot: <jeanfred> Triggered a manual jsub update_monuments.sh (T244213)
  • 08:45 wm-bot: <lokal-profil> Killed update_monuments job (T244213)

2019-11-02

  • 14:29 wm-bot: <jeanfred> Triggering a new full database update as the last one crashed and the DB is empty

2019-10-12

  • 22:17 wm-bot: <jeanfred> Deleted old GridEngine jobs from Oct 7 (jstop update_monuments jstop categorize_images)

2019-10-10

  • 19:02 wm-bot: <lokal-profil> restarted webservice

2019-09-20

  • 19:49 wm-bot: <jeanfred> Added crontab (*/3 - every three minutes) for T195341

2019-09-18

  • 20:52 wm-bot: <jeanfred> check_emailable_users.py -category:"Images_from_Wiki_Loves_Monuments_2019" -delta:540 -notify (T195341)

2019-09-17

2019-09-09

2019-09-08

2019-09-03

2019-09-02

2019-09-01

  • 12:59 JeanFred: Deploy latest from Git master: 52b2f4d

2019-08-31

2019-08-30

  • 16:46 JeanFred: Deploy latest from Git master: 809e004

2019-08-29

2019-08-26

2019-04-12

  • 10:04 JeanFred: Started a new harvest to better investigate T180833

2019-03-22

  • 21:40 Lokal_Profil: Manually starting harvesting job to ensure migration (T216364) worked
  • 20:52 Lokal_Profil: Starting Stretch migration

2018-12-01

  • 14:23 JeanFred: Service was returning 502 (even though `webservice status` was indicating "Your webservice of type php5.6 is running"), did a `webservice restart`.

2018-10-04

2018-10-02

2018-09-20

2018-09-13

2018-09-10

2018-09-09

2018-09-05

2018-09-04

  • 06:36 JeanFrd: Edit crontab to run `update_monuments` with ` -mem 2000m` (see T203417)
  • 06:35 JeanFrd: Submit `update_monuments` manually with ` -mem 2000m` (see T203417)
  • 06:26 JeanFred: [yesterday] Deploy latest from Git master: 68de059, bffe358, d8c9e7e
  • 06:26 JeanFred: [two days ago] Bump the pywikibot checked out on the server in ~/pywikibot (git fetch && git checkout 3.0.20180823)

2018-09-02

  • 21:21 Lokal_Profil: Manual restart of update_monuments.sh

2018-09-01

2018-08-24

2018-08-20

2018-08-13

2018-07-31

2018-07-26

2018-07-22

2018-07-20

2018-02-25

2017-11-17

  • 21:31 JeanFred: Reverted to old database replicas (via `git reset HEAD~1 && git stash`) as part of T180833 investigation
  • 19:30 JeanFred: Started a new harvest to better investigate T180833

2017-11-16

2017-10-26

2017-09-28

2017-09-26

  • 23:21 JeanFred: Deploy latest from Git master: 0dddaf5
  • 23:09 JeanFred: Deploy latest from Git master: 8248ff4 (T176200)
  • 11:13 JeanFred: Deploy latest from Git master: 2828a0f (T117327)
  • 09:52 JeanFred: Deploy latest from Git master: 263ccee (T174556)

2017-09-25

2017-09-24

2017-09-23

2017-09-22

2017-09-21

2017-09-19

2017-09-18

  • 13:58 JeanFred: Killed `update_monuments` job as it had been stuck for 3 days on the `Update monuments_all table` step.

2017-09-17

  • 22:36 Lokal_Profil: Service down again ("backend is overloaded" in error.log). Tried another webservice restart
  • 18:28 JeanFred: Deploy latest from Git master: b3c3ab0 (T174871)
  • 18:28 JeanFred: Restarted the webservice − Yarl notified us around 10 hours ago that API was inaccessible. `webservice restart` fixed the issue.

2017-09-14

2017-09-13

2017-09-06

2017-09-04

2017-09-01

2017-08-25

2017-08-22

2017-08-09

2017-08-02

2017-05-19

2017-04-20

2017-02-17

2017-02-12

2017-01-25

  • 18:58 JeanFred: Deploy latest from Git master: 0810246 (T156139)
  • 13:30 JeanFred: Deploy latest from Git master: 8c75342

2017-01-09

2016-12-21

2016-11-30

2016-11-08

  • 16:17 JeanFred: Deploy latest from Git master: a70289c (T149258), c7eb06a, b5aeb29, 828e309

2016-10-26

  • 23:03 JeanFred: Deploy latest from Git master: c943e89, e3ba148, 063f9e2 (T148772 & T148773), 1db9f42, fe119e1, de2e590, 82147b8, f6ff350 & 63a1819 (T140795)

2016-10-25

  • 16:44 JeanFred: Deploy latest from Git master: c943e89, e3ba148, 063f9e2 (T148772 & T148773), 1db9f42, fe119e1, de2e590, 82147b8, f6ff350 & 63a1819 (T140795)

2016-10-19

  • 20:42 JeanFred: Deploy latest from Git master: 213b4a9 (T132644)
  • 20:15 JeanFred: Deploy latest from Git master: 9f67e94, c64c22e, a21517e

2016-10-12

  • 09:15 JeanFred: Deployed master from Git: dce7434, 8bfba2a (T145333), faa356e, f87cb70 (T145574), 64ea088 (T143573), 8187df6 (T132641), 26171fe (T138633)

2016-09-30

  • 19:22 JeanFred: Deployed latest from Git: 08ea28d, fb856b4, 867a229, a454375

2016-09-29

2016-09-22

  • 16:29 Lokal_Profil: Deployed latest from Git, 2969082, cb1b318 (T114166)
  • 08:00 Lokal_Profil: Deployed latest from Git, f439ac4, fab9a23, ff37691, 241a27a, 0a45ad0, 1a526a5, 23ce89e (T144772)

2016-09-21

  • 13:41 Lokal_Profil: Reverting local changes to categorize_images.py (T146278). The CI mechanisms are there for a reason

2016-09-16

  • 13:04 JeanFred: Deactivate categorisation for ('it', 'it') as some Wikimedia Commons users are unhappy with it.
  • 09:40 Lokal_Profil: Added ka.wikipedia to pywikibot user_config

2016-09-02

  • 16:24 JeanFred: Deployed latest from Git: 1603efb (T142570)

2016-08-31

  • 20:27 JeanFred: Pulled latest pywikibot (branch 2.0) from Git: 8 commits, including fix for T144438.
  • 20:02 JeanFred: Deployed latest from Git: 22496d6 (T143481), c964df5, 00ccf8a, 1891ee0, 4458eeb, 55fea41

2016-08-09

  • 07:39 Lokal_Profil: Deployed latest from Git, 768b3ac, 30e33ca, 8d7de41 (T141505)

2016-08-08

  • 09:06 Lokal_Profil: Updating pywikibot (37 commits)

2016-08-01

  • 10:56 JeanFred: Deployed latest from Git: 6763dbb and dadf805 (T141757)
  • 07:54 Lokal_Profil: (correction to last line) Deployed latest from Git, 5fe42fe (T111618), 1ec3530, 9a630b5 (T139258)
  • 07:53 Lokal_Profil: Deployed latest from Git, 5fe42fe (5fe42fe), 1ec3530, 9a630b5 (T139258)

2016-07-26

  • 13:42 JeanFred: Deployed latest from Git: 0c848e6 (T140488)
  • 08:30 Lokal_Profil: Add sq.wikipedia to pywikibot user_config

2016-07-25

  • 13:30 JeanFred: Deployed latest from Git: ff61234, 112c14f, 7e98153, b614e78, aafc788 & 3222df9 (T140795), baad88c, ac388ee

2016-07-13

  • 09:08 Lokal_Profil: Deployed latest from Git, 38363cb, 6117b13 (T139267), 96af0dc, 20d6da6

2016-07-07

  • 22:26 JeanFred: Deployed latest from Git: 1022e80, ad29828, 7ab4bcf, 48c96b4 (T139580), f29995d (T138633), 92f9234
  • 22:22 JeanFred: Deployed latest from Git: 6e6cc59

2016-07-06

  • 14:28 JeanFred: Deployed latest from Git: 6e6cc59
  • 11:23 yuvipanda: restarting tool with 'webservice stop' 'webservice --backend=kubernetes start'
  • 11:16 JeanFred: Deployed latest from Git: 3323de1 and 6d20267 (T138513)

2016-07-04

  • 00:22 JeanFred: Deployed latest from Git: 2f9024e

2016-07-03

  • 15:16 JeanFred: Deployed latest from Git: 76383f9
  • 14:44 JeanFred: Deployed latest from Git: 1fdeff6
  • 13:01 JeanFred: Deployed latest from Git: 0154a31, 9a4d05b, 4b6c343, 39c5409 (T138633)

2016-06-30

  • 18:43 JeanFred: Deployed latest from Git: 795a396 (T136351), 4c8d9e3, e09e3d7 (T138519), 7840307, 4bdddd0 (T138606)

2016-06-24

  • 22:48 Lokal_Profil: Recreated source tables (T138606)

2016-06-23

  • 15:33 JeanFred: Added column wd_item to monuments_all, by copying monuments_all to tmp, alter table, and rename back to avoid locks. (T55808)
  • 14:33 JeanFred: Running 'ALTER TABLE `monuments_all` ADD COLUMN `wd_item` varchar(255) DEFAULT NULL;' ; taking a while...
  • 14:27 JeanFred: Stopped webservice, restarted and tying on trusty (`webservice --release=trusty start`)
  • 14:24 JeanFred: Monuments API currently down because of PHP 5.5 syntax, and host running 5.3
  • 14:01 JeanFred: Deployed latest from Git: 4030533, bb95d23 (T55808), bd96bbd (T138377), 0a3247d, be9b1a9 (T134764)
  • 09:42 multichill: Added the fa Wikipedia account for Pywikibot. This should fix the broken unused image job

2016-06-22

  • 16:56 JeanFred: Deployed latest from Git: 76c6dd6c (T138377)
  • 15:42 JeanFred: Deployed latest from Git: 4be4f04, c280649, d667e19 (T136566 & T137543), e867c45, 0a09a20

2016-06-10

  • 14:09 JeanFred: Deployed latest from Git: 74d9086
  • 10:30 JeanFred: Deployed latest from Git: d25eda5 (T136704)

2016-06-06

  • 15:46 JeanFred: Deployed latest from Git: b77b7c2 (T137096)

2016-06-03

  • 14:00 JeanFred: Deployed latest from Git: 5ca0fdc, 688141a, 1af3b3d, 0262786, d812cdf (T134565)

2016-05-23

  • 12:50 Lokal_Profil: Deployed latest from Git, 50915bf (T55688)

2016-05-18

  • 10:50 JeanFred: Deployed latest from Git: 39780e2, 977c07f, 5f4532c, b7b297b (T135502 & T55688), 476267f (T39422)

2016-05-12

  • 13:04 JeanFred: Deployed latest from Git: ebcd48c (T134727)

2016-05-09

  • 11:15 JeanFred: Deployed latest from Git: 4f82b32 (T134728)

2016-05-06

  • 19:47 Lokal_Profil: Deployed latest pywikibot-core/2.0 from Git
  • 19:26 Lokal_Profil: Deployed latest from Git, a724279 , d9ae73d (reverts 766d814 )
  • 18:42 Lokal_Profil: Deployed latest from Git, 2d3ee40 (T39974), 766d814, e2fac07 and d2c242a (T39422)
  • 14:58 JeanFred: Deployed latest from Git: e5a9f01 and d509343 (T134567)
  • 12:15 JeanFred: Deployed latest from Git: db46042, c765e76, b5a731a, 7c27207, d4de720 (T134236), e7823ab & c83003b (T132647), 615ab28

2016-04-20

  • 13:01 JeanFred: Deployed latest from Git, 48bce77 and dfbff9b (T132029)

2016-04-01

  • 22:51 multichill2: JeanFred did a git pull for Phab:T131344 and others

2016-03-31

  • 09:14 multichill: Commented out the Russian Wikipedia in user-config.py for Phab:T131344

2016-03-16

2015-08-30

  • 14:38 multichill: Made local change to unused_images.py to get it to work, see https://phabricator.wikimedia.org/T110829
  • 09:14 multichill: Updated ~/pywikibot to latest version, but still getting a FamilyMaintenanceWarning

2015-08-22

  • 13:35 JeanFred: After backporting all local changes to Gerrit, updating local checkout to latest Git version.

2015-07-15

  • 16:50 JeanFred: Checked out pywikibot-core

February 23

December 21

September 20

August 27

  • 18:05 multichill: Added Oren to the project

July 10

June 29

  • 21:43 multichill: I put the RCE mysql conversion User:Akoopal made in ~/rce-nl-data . Still need to import it in Mysql to be useful. Data is CC0
  • 19:30 multichill: Web service was down for all accounts. Back up and running. Api seems to have been down from 19:30 to 21:15 (Amsterdam time)
  • 13:36 multichill: Burned the old ~erfgoed account on the Toolserver and uploading the backup to ~/toolserver_backup/

June 19

June 15

  • 11:17 multichill: Did some hacks with Krinkle to get i18n working(ish) again (api.php and html formatters). Still need to commit it

June 14

June 7

  • 16:57 multichill: While updating documentation I found https://phabricator.wikimedia.org/diffusion/GWLA/ . Should probably be dropped, everything is in https://phabricator.wikimedia.org/diffusion/THER/history/
  • 16:12 multichill: http://toolserver.org/~erfgoed/ now redirects to http://tools.wmflabs.org/heritage/ . Didn't move everything so that might give some 404's
  • 16:06 multichill: prox_search completed without problems. update_monuments.sh should now run without failures.
  • 15:52 multichill: symlinked ~/prox_search, fixed path (need to commit that), create_table_prox_search.sql , doing manual run
  • 15:35 multichill: Had to increase memory for statitics to 512M. Still need to commit that. jsubbed build_stats_test again and it finished with Memory usage: 396588928
  • 15:04 multichill: Symlinked ~/public_html/maintenance and create tables statistics and statisticsct. jsubbed build_stats_test to test it
  • 14:49 multichill: Fixed populate_adm_tree.php and populated the table. Still need to commit it

June 4

June 1

  • 20:18 multichill: Set up cron to run the update_monuments job every night. Some parts of it will still fail.
  • 20:05 multichill: Some tweaks in https://gerrit.wikimedia.org/r/136683 database is filled. Api is working (admintree and statistics still missing)
  • 17:14 multichill: Updated ~/bin/create_all_monuments_tables.sh and created 105 tables. Fired up update_database.py to fill the database
  • 17:01 multichill: Pull pywikibot (compat) and heritage. Symlinked it and setup the bot
  • 16:46 multichill: Moved erfgoedbot, public_html & pywikipedia to ~/old/. to make room
  • 16:41 multichill: Fixed ~/.database.inc , still have to do the i18n part
  • 16:35 multichill: Cleaned out some code in https://gerrit.wikimedia.org/r/136649 and merged it
  • 16:18 multichill: Created the s51138__heritage_p database
  • 16:16 multichill: Replaced the .my.cnf with the right credentials
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Tools.heritage/SAL&oldid=2245756"