Nova Resource:Wikistats/SAL

From Wikitech

2024-04-01

  • 17:23 mutante: - deleting instance bwplanet

2024-02-23

  • 20:20 RhinosF1: testing

2024-02-08

  • 18:24 mutante: running '/usr/lib/wikistats/update.php mw autofixit' which tries to magically fix stats URLs that are 301 and return a new location .. but it throws an error because then there are duplicate entries which the DB doesn't allow

2023-12-30

2023-11-04

  • 09:41 RhinosF1: fandom/wikia migration at about 20k/70k checked

2023-11-03

  • 20:30 RhinosF1: copied update.php to updatewi.php and put it in a screen
  • 19:53 RhinosF1: hacked update.php on disk with MR 5 on gitlab and started updating wikia table
  • 19:34 RhinosF1: MariaDB [wikistats]> ALTER TABLE wikia ADD COLUMN si_generator VARCHAR(255);

2023-10-27

  • 16:57 RhinosF1: /usr/lib/wikistats/update.php wp prefix fon T347943
  • 16:56 RhinosF1: /usr/lib/wikistats/update.php wp prefix fon T347943
  • 16:55 RhinosF1: insert into wikipedias (prefix, lang, loclang, method) values ("fon", "Fon", "fɔ̀ngbè", 8); -- T347943

2023-09-12

  • 20:18 RhinosF1: final update today (to update versions list)
  • 20:12 RhinosF1: END maintenance; kernel upgrade and database optimised
  • 20:08 RhinosF1: START: maintenance, expect outages
  • 20:04 RhinosF1: deploying to drop lxde
  • 09:41 RhinosF1: drop wikia where http = 404 T215534
  • 08:09 RhinosF1: drop referata table after dumping it T262148
  • 08:07 RhinosF1: drop referata table after dumping it
  • 08:04 RhinosF1: deploy latest code for T262148 again
  • 08:00 RhinosF1: deploy latest code for T262148

2023-08-30

  • 11:00 RhinosF1: create langdata table from wikipedias T345235
  • 09:29 RhinosF1: add tlywiki -- T345174
  • 09:23 RhinosF1: as suwikisource and blkwkitionary T343552 T343546

2023-08-15

  • 16:19 RhinosF1: delete wikistats-bullseye VM

2023-08-01

  • 18:25 RhinosF1: switch main proxy to bookworm (aokoth) and disable everything so old one is RO (rhinosf1)

2023-07-28

  • 16:32 RhinosF1: drop a wiki from mediawikis table that had domain resolving to 127.0.0.1

2023-07-26

  • 20:48 RhinosF1: wikistats-new.wmcloud is running bookworm as a beta kinda working, expected double traffic from wikistats scraping while test happens
  • 20:17 RhinosF1: wikistats-bookworm: deploy code, start importing db
  • 19:55 RhinosF1: wikistats-bookworm: update ram+cpu with arnold, bootstrapping mysql
  • 17:55 RhinosF1: setup wikistats-new proxy and add role to bookworm instance
  • 17:42 RhinosF1: add wikifunctions to mariadb table & /usr/lib/wikistats/update.php wx prefix wikifunctions
  • 16:28 RhinosF1: /usr/lib/wikistats/update.php wt prefix btm - T336119
  • 16:28 RhinosF1: MariaDB [wikistats]> insert into wiktionaries (prefix, lang, loclang, method) values ("btm", "Mandailing", "Saro Mandailing", 8); - T336119
  • 16:15 RhinosF1: MariaDB [wikistats]> insert into wiktionaries (prefix, lang, loclang, method) select prefix,lang,loclang,method from wikipedias where prefix="btm"; - T336119

2023-07-05

  • 19:16 RhinosF1: dropped all non http 200 from miraheze table
  • 18:58 RhinosF1: fix foundationwiki, ee/et.wikimedia, pt.wikimedia, wikimedia.at
  • 18:43 RhinosF1: fix statsurl for wikitech
  • 18:24 RhinosF1: deleted broken tests
  • 18:14 RhinosF1: /usr/lib/wikistats/update.php wp prefix gpe T335991
  • 18:14 RhinosF1: MariaDB [wikistats]> insert into wikipedias (prefix, lang, loclang, method) values ("gpe", "Ghanaian Pidgin", "Kroo brofo", 8) T335991

2023-06-21

  • 18:30 mutante: - adding user RhinosF1 as reader

2023-06-20

  • 23:08 mutante: - added user Aokoth as reader and member

2023-05-01

  • 17:50 mutante: - running "/usr/lib/wikistats/update.php mw autofixit" which means "go trough mediawikis table, take all wikis that got a 301 last time they tried to talk to API, see where you get redirected to and if it looks like a valid api.php then use the new URL and update the database".

2021-12-14

  • 23:12 mutante: - removing mysql root password to local mariabdb on wikistats-bullseye. data is meant to be public. getting ssh shell in the project is enough access control

2021-12-10

  • 16:57 mutante: added api.wikimedia.org and thankyou.wikipedia.org to table of "wikimedia special wikis"
  • 16:35 mutante: shutting down deprecated buster instance "wikistats-wild-tiger". replaced by "wikistats-bullseye" on.. you guessed it.. bullseye. @user jem your scripts have been copied from your home dir to /usr/local/bin/wikistats/

2021-10-22

  • 22:36 mutante: - deleted simplelamp temp test instance
  • 22:36 mutante: - creating new instance wikistats-bullseye for testing upgrade to next release

2021-10-04

  • 21:22 mutante: /usr/local/bin/wikistats/import_miraheze.sh T292369

2021-01-08

  • 23:47 mutante: - shutting down and removing instance wikistats-dancing-goat - backup stored on -wild-tiger
  • 23:35 mutante: deleting web proxy wikistats-old

2020-12-10

  • 00:05 mutante: re-enabling puppet (disabled without reason) on wikistats-wild-tiger

2020-12-07

  • 22:25 mutante: - replacing all cron jobs with systemd timers - gerrit:645455 - T265138

2020-09-04

  • 23:57 mutante: fixed updates of gamepedia wikis

2020-07-30

  • 19:41 mutante: wikistats-dancing-goat - cron service was broken for unknown reason - systemctl start cron - brought it back

2020-07-27

  • 19:37 jem: added ary.wikipedia

2019-11-22

  • 19:10 paladox: deleting instance wikistats-greyhound
  • 19:09 mutante: shutting down instance wikistats-greyhound

2019-10-02

2019-09-30

  • 19:18 mutante: - created new instance dancing-goat with buster as backup and to replace stretch
  • 19:17 mutante: - deleted old instances T128642 and T21008

2019-04-18

  • 19:05 bd808: deleting some project- and prefix-wide puppet config to try to get puppet running again on VMs (re-logged for andrewbogott)

2019-02-08

  • 22:52 mutante: - creating throw away instance to test if T128642 is still true or not
  • 22:50 mutante: - shutting down and deleted instance T210008 once made for testing krypton upgrade but broken somehow

2018-12-14

  • 18:39 mutante: - ran deploy-wikistats deploy, copied .xml files, migrationg to new region should be all done now
  • 18:25 mutante: - replaced wikistats-kraken (eqiad) with wikistats-greyhound (eqiad-r), new stretch image, copied mysql data over, deleted and recreated web proxy
  • 14:24 andrewbogott: migrating project to eqiad1-r

2018-11-20

  • 22:02 mutante: deleted instance wikistats-cowgirl (jessie), replaced after > 3 years

2018-11-16

  • 01:58 mutante: - added new projects: punjabi.wikimedia, yue.wiktionary, li.wikinews and shn.wikpedia
  • 01:19 mutante: - shut down instance cowgirl (jessie), replaced by kraken (stretch)
  • 00:28 mutante: deleting instance tacotuesday2 - not used

2018-11-15

  • 04:16 mutante: switched backend of web proxy to wikistats-kraken (stretch upgrade), migrated db dump from jessie instance cowgirl, deleted extra webproxy

2018-11-14

  • 17:04 mutante: - mysql server currently down, replacing mysql with mariadb and puppetizing, maintenance mode until later today

2018-01-09

  • 00:10 mutante: instance -kraken, upgrade php7 and mariadb packages

2017-10-17

  • 18:53 paladox: wikistats-cowgirl upgrading linux-meta-4.9 as last package to upgrade.
  • 18:28 mutante: wikistats-cowgirl installing apache,systemd, misc upgrades

2017-06-23

  • 00:01 mutante: - added kbp.wikipedia and pt.wikimedia

2017-05-19

  • 22:07 mutante: fixed puppet issues, puppetized db pass and grants to bootstrap instances, apt-get upgrading wikistats-petcow (jessie), testing stretch with separate instance -octopus ..

2017-05-18

  • 19:17 mutante: - deployed code change that removes ALL mysql_ function remants (they have been removed in PHP7/stretch). now using PDO to connect to db.

2017-05-15

  • 22:15 mutante: launching new instance wikistats-octopus with experimental stretch image

2017-04-26

  • 03:27 mutante: added new Wikipedia "dty" - Doteli (T160947)

2016-08-15

  • 02:17 mutante: deleted 'nostalgia' from wikipedias table per request (it's in the 'specials' table already)
  • 01:52 mutante: major cleanup of backend data. deleted and fixed hundreds (redirects, duplicates, broken wikis, convert to API, ..)

2016-07-12

  • 21:42 mutante: deleted wikistats-southpark instance to free resources
  • 21:40 chasemp: nova start wikistats-southpark.wikistats.eqiad.wmflabs
  • 18:33 mutante: shutting down instance wikistats-southpark, lastlog said not used since February

2015-07-15

  • 02:56 mutante: - apt-get upgrade

2015-07-14

  • 23:48 mutante: added more missing projects, pnb.wikt, ca.wm, cn.wm ..
  • 23:47 mutante: built and installed packaged 2.10
  • 23:02 mutante: add missing Persian Wikivoyage (fa)

September 18

  • 23:11 mutante: - package install problem due to bug 70981

August 20

  • 23:52 mutante: - so that now we have puppet status OK again (missing puppetized db setup)
  • 23:50 mutante: - deleted old proxy, created new one, switched backend over to wikistats-petcow instance

August 19

  • 21:14 mutante: - puppet broken, Apache setup, DB setup, DPKG, attempting to revive on fresh instance wikistats-petcow

March 28

  • 16:14 mutante: - migrated over to eqiad instance
  • 15:45 mutante: - deleted instance wikistats-mod
  • 15:45 mutante: - deleted instance wikistats-history-01

February 28

  • 04:57 mutante|away: - turned into puppet module

February 24

  • 21:25 mutante: giving back public IP
  • 21:18 mutante: putting it behind web proxy instead
  • 20:59 mutante: disabling SSL, certificate has been revoked
  • 20:59 mutante: restored disapeared IP and DNS entries

January 6

  • 13:00 mutante: - metapedias - fix broken table header, add language columns
  • 12:36 mutante: - metapedias (error 302) - sk.metapedia, et.metapedia, fix stats URLs (yea, use /m/ instead of /w/ now :P)
  • 12:20 mutante: - editthis (error 991) - delete 'bridge' (readapi denied), delete 'kanji' (No db present), delete 'kickipedia' (no reply), delete 'lost20' (no reply)

December 8

  • 19:36 mutante_: fix fetching of extended info, add new API fields si_langconversion,si_linktrail,si_linkprefix,si_titleconversion. run update on wikis without names to give them one from si_sitename
  • 19:30 mutante_: (mw) 10 more converted from Special page to API using different ways to guess the api.php URL. deleted a couple broken ones
  • 17:51 mutante: (mw) ran conversion script attempting to find ancient wikis that meanwhile may have api.php. 10 wikis succesfully converted (out of 190)
  • 17:21 mutante: (mw) fixed: irc-wiki.org, wiki.electrorama.com
  • 17:07 mutante: (mw) id 143 switch to API, battlestarwiki.org wikis, fixed es,fr,zh,deleted duplicates de,en..
  • 16:49 mutante: - referata wikis, deleted 28 closed wikis that became 404

December 7

  • 21:05 mutante: - various fixes with some broken updates in wiktionary and mediawiki table, remaining ones with error code "991" means the remote site has database issues, even if it looks like it works in browser, the script gets internal_api_error_DBConnectionError or similar
  • 21:03 mutante: (wx) - chapter wikis - removed because they all don't use mediawiki anymore (sigh!?) - wikimedia.fr, wikimedia.ch, wikimedia.org.ve, wikimedia.org.il, wikimedia.org.ar ..(less and less chapters actually use mw)
  • 21:02 mutante: (wx) - chapter wikis - fixed URLs due to site redesigns: wikimedia.de, wikimedia.org.ph, pa.us.wm -> pa-us.wm

September 11

  • 00:19 mutante: switching from public IP to YuviProxy

September 4

  • 19:43 mutante: - fixed stats updates for a couple hundred external mediawikis, by adding a trim() :p for some reason some API results have whitespace

July 30

  • 02:24 mutante: added missing el.wikivoyage, fixed wikivoyage timestamps/updates, running wiki-site update, add 1.22wmf12 to good versions

March 27

  • 23:49 mutante: - add new 'he' and 'uk' wikivoyages

March 26

  • 02:06 mutante-away: - installing wikistats package 2.2
  • 00:57 mutante: - fix sorting in wikimedias table
  • 00:22 mutante: - adding wikivoyage to wikimedias_html table
  • 00:14 mutante: - updating mw versions of wmf wikis

October 9

  • 18:56 mutante: - adding several W3C wikis to mediawiki table

October 4

  • 22:52 mutante: added docs and how to build the .deb

October 3

  • 00:58 mutante: - upgrading mariadb packages (yeah:)

October 2

  • 18:28 mutante: fixed language name for arc.wp "Assyrian Neo-Aramaic" -> "Aramaic"
  • 18:22 mutante: fixed updates for "vep" and "nso" wikipedias (did not try API method).
  • 18:15 mutante: added wikimedias_csv, fixed links in wikimedias_html, updates WMF wiki versions, ...

July 13

  • 11:05 mutante: - switched to use just a single role class & puppetmaster::self

June 22

  • 13:30 mutante: - stopped puppet - misc::wikistats is gone / 404 momentarily.
  • 13:11 mutante: switching wikistats-01 instance to role::labs-mysql-server
  • 12:53 mutante: switching wikistats-01 instace from mariadb classes to generic::mysql::server

May 29

  • 12:24 mutante: foo

May 16

  • 18:51 mutante: - updating mw versions for wmf wikis (heh, while deployment is going on)

May 11

  • mutante: added new member MarkAHershberger
  • mutante: gave bastion and wikistats shell access to Mschon
  • mutante: added WSA script for Robih

May 10

  • mutante: added new project member Mschon

May 9

  • 13:43 mutante: granting select on wikistats.* to robih @ localhost
  • 12:03 mutante: running puppet and missing /home/robih

February 10

  • 17:38 mutante: fixed puppet runs, generic::webserver -> webserver