Nova Resource:Tools.notwikilambda/SAL

From Wikitech

2023-09-29

  • 12:19 wm-bot: <lucaswerkmeister> kubectl delete cronjob restart (T314880)

2023-04-16

  • 17:27 wm-bot: <lucaswerkmeister> sql tools 'DROP DATABASE s54524__mediawiki' # T314880

2023-02-02

2022-11-01

  • 22:03 wm-bot: <lucaswerkmeister> deleted 1.76 GiB of npm cacache and composer cache (cc T314880)

2022-10-30

  • 11:00 wm-bot: <lucaswerkmeister> lowered notwikilambda quotas: count/deployments.apps from 5 to 3, services from 4 to 1 (T321990)
  • 10:58 wm-bot: <lucaswerkmeister> kubectl delete service function-evaluator function-orchestrator pygments-server # T314880
  • 10:34 wm-bot: <lucaswerkmeister> kubectl delete deployment function-evaluator function-orchestrator pygments-server (T314880)
  • 10:34 wm-bot: <lucaswerkmeister> kubectl delete deployment function-evaluator function-orchestrator pygments-server
  • 10:22 wm-bot: <lucaswerkmeister> start webservice again (PHP 7.4 now) with new lighttpd config to rewrite everything to a very bare index.html explaining the shutdown (T314880); partial SQL dump available in public_html too
  • 10:04 wm-bot: <lucaswerkmeister> stopped webservice, wiki is being retired (T314880)

2022-07-03

  • 15:58 wm-bot: <lucaswerkmeister> deployed e5143b3012 (add nfs-mounted selector, contributed by taavi) using `kubectl edit`

2022-06-03

  • 19:42 wm-bot: <lucaswerkmeister> started webservice again
  • 19:42 wm-bot: <lucaswerkmeister> deleted update deployment again, update.php also crashed in k8s and I don’t want the code to update while the schema is frozen
  • 19:36 wm-bot: <lucaswerkmeister> recreated update deployment, update.php crashed within webservice shell and the update deployment should have more memory
  • 19:27 wm-bot: <lucaswerkmeister> stopped webservice and update deployment, something’s broken
  • 19:23 wm-bot: <lucaswerkmeister> updated pygments-server to f2c760810b (minor bugfix in pygmentize wrapper script)

2022-04-06

  • 20:28 wm-bot: <lucaswerkmeister> deployed f6e6dc49e1 (allow function-evaluator startup up to 20 minutes)
  • 19:20 wm-bot: <lucaswerkmeister> resumed automatic updates of PluggableAuth (T299934)
  • 18:53 wm-bot: <lucaswerkmeister> updated to PluggableAuth 6.0 (T299934)

2022-02-25

  • 09:04 wm-bot: <lucaswerkmeister> git -C public_html/w/extensions/PluggableAuth/ checkout 520da2f # T299934

2022-02-13

  • 14:22 wm-bot: <lucaswerkmeister> kubectl create -f ~/k8s/update/deployment.yaml # resume automatic updates except for extensions/PluggableAuth (T299934)
  • 14:02 wm-bot: <lucaswerkmeister> Checked out WSOAuth master again (T298351)

2022-02-06

2022-01-24

  • 17:34 wm-bot: <lucaswerkmeister> git -C ~/public_html/w/extensions/PluggableAuth/ checkout cdf73daf11^ # T299934
  • 17:25 wm-bot: <lucaswerkmeister> git -C ~/public_html/w/extensions/PluggableAuth/ checkout 8b278afaca # T299934
  • 17:24 wm-bot: <lucaswerkmeister> kubectl delete deployment update # T299934

2021-11-24

  • 18:01 wm-bot: <lucaswerkmeister> renamed $wgWikiLocation to $wgWikiLambdaWikiAPILocation in local settings T296421, and also changed it to internal hostname (notwikilambda:8000), no need to go through the Toolforge proxy
  • 17:50 wm-bot: <lucaswerkmeister> recreated function-evaluator deployment (still on commit 8731f2ff6e, no changes) T296421
  • 17:47 wm-bot: <lucaswerkmeister> ran `npm ci` in function-evaluator (in webservice node12 shell), for some reason node_modules/limitation was on 0.2.2 despite package-lock.json having 0.2.3 (T296421)
  • 17:39 wm-bot: <lucaswerkmeister> deleted function-evaluator deployment T296421

2021-10-19

  • 20:56 wm-bot: <lucaswerkmeister> previous log messages all related to T293828
  • 20:41 wm-bot: <lucaswerkmeister> added $wgMemoryLimit = '1G'; to LocalSettings.php
  • 20:32 wm-bot: <lucaswerkmeister> added mem: 2G to ~/service.template and stop+started webservice
  • 20:26 wm-bot: <lucaswerkmeister> added git submodule update to ~/update script
  • 20:23 wm-bot: <lucaswerkmeister> git -C ~/public_html/w/extensions/WikiLambda/ submodule update

2021-10-06

  • 20:24 wm-bot: <lucaswerkmeister> mount only function-orchestrator source into pod (8731f2ff6e), working around apparent “toolforge: tool” issue

2021-09-25

  • 17:13 wm-bot: <lucaswerkmeister> cleaned up old replicasets: kubectl delete rs $(kubectl get rs -o json | jq -r ".items |.[] | select(.status.replicas == 0) | .metadata.name")
  • 16:45 wm-bot: <lucaswerkmeister> updated pygments-server to Python 3.9 (cef7948b15)
  • 16:37 wm-bot: <lucaswerkmeister> switched pygments-server from web to base image (25f8a10fa0)
  • 16:36 wm-bot: <lucaswerkmeister> switched function evaluator and orchestrator from web to base image (8424307b5d)
  • 16:29 wm-bot: <lucaswerkmeister> shortened periodSeconds of JS container probes from 15 to 5 seconds (3d658c10ba)
  • 16:18 wm-bot: <lucaswerkmeister> added --no-save to npm install in function evaluator and orchestrator (266c550663) and manually reset the package-lock.json in both repos
  • 11:36 wm-bot: <lucaswerkmeister> deployed pygments-server 1ae6e19920 (preserve boolean status of exit code)
  • 10:58 wm-bot: <lucaswerkmeister> deployed new pygments-server that works with latest SyntaxHighlight (i.e. supports input files), and restored SyntaxHighlight to latest master; it turns out yesterday’s code worked perfectly fine, MediaWiki just cached a broken lexers list in APCu, which didn’t get purged by action=purge, and also didn’t show up in shell.php because the APCu cache is disabled in cli blaarggghhhh

2021-09-24

  • 18:58 wm-bot: <lucaswerkmeister> tried to deploy new pygments-server with support for latest SyntaxHighlight; not quite working yet, but as pages with SyntaxHighlight still work (just without highlighting), I’m okay to leave it at this for now, and hopefully resume tomorrow

2021-09-22

  • 19:57 wm-bot: <lucaswerkmeister> deleted and recreated `update` deployment, it was stuck unable to roll out a new replicaset due to quota; I’m not sure where the new replicaset came from, maybe I accidentally rollout-restarted it?

2021-09-21

  • 23:40 wm-bot: <lucaswerkmeister> set pygments-server memory limits to 256Mi (3cea8f6a18)
  • 19:29 wm-bot: <lucaswerkmeister> undeployed workaround for T291325, seems to be fixed

2021-09-20

  • 21:23 wm-bot: <lucaswerkmeister> successfully rolled out node12 for function-orchestrator too
  • 21:19 wm-bot: <lucaswerkmeister> successfully rolled out node12 for function-evaluator after changing `npm ci` to `npm install` (ecfa5ab9ab)
  • 21:00 wm-bot: <lucaswerkmeister> tried to deploy d028cb3667 (switch to node12) for function-evaluator but the new pod never came up so I undid the rollout; will consider further steps
  • 20:50 wm-bot: <lucaswerkmeister> set cronjob successfulJobsHistoryLimit to 0 (0aa054b391)
  • 20:40 wm-bot: <lucaswerkmeister> `git -C ~/www/js/function-orchestrator/ config pull.rebase true` ditto
  • 20:40 wm-bot: <lucaswerkmeister> `git -C ~/www/js/function-evaluator/ config pull.rebase true` to silence `git pull` warnings (presumably due to newer git in node12 container)

2021-09-19

  • 16:47 wm-bot: <lucaswerkmeister> changed restart cronjob to 9:30 and 21:30 to avoid congestion at midnight/noon
  • 16:39 wm-bot: <lucaswerkmeister> (and reenabled SyntaxHighlight_GeSHi)
  • 16:39 wm-bot: <lucaswerkmeister> rolled back SyntaxHighlight_GeSHi to commit 580ce3425f, because I2e82e5aa2a is incompatible with pygments-server (provides input as file instead of stdin)
  • 15:48 wm-bot: <lucaswerkmeister> temporarily disabled SyntaxHighlight extension due to issues with the Pygments server
  • 15:34 wm-bot: <lucaswerkmeister> improved automatic updates of function orchestrator and evaluator, should work without downtime now
  • 10:47 wm-bot: <lucaswerkmeister> configured $wgFooterIcons if notwikilambda-configure-wgFooterIcons URL parameter isn’t set to false, working around T291325

2021-08-25

  • 21:05 wm-bot: <lucaswerkmeister> namespace-protect ZObject and ZObject talk namespaces so nobody (not even sysops) can edit them – there should be no reason to use that namespace after T289682 and I assume WikiLambda will soon stop defining it
  • 20:43 wm-bot: <lucaswerkmeister> removed namespace protection from main namespace, now that move of ZObjects to main namespace is completed (T289682)
  • 20:19 wm-bot: <lucaswerkmeister> deployed new versions of function orchestrator+evaluator deployments which don’t skip the `npm ci` step (since the old override of the kad package doesn’t seem to be necessary anymore)
  • 19:46 wm-bot: <lucaswerkmeister> removed $wgNamespacesToBeSearchedDefault from LocalSettings.php now that the ZObject namespace is no longer used
  • 19:43 wm-bot: <lucaswerkmeister> updated and restarted function-orchestrator and function-evaluator (they were ancient, I’ll have to check why the update script doesn’t seem to update them)
  • 19:24 wm-bot: <lucaswerkmeister> kubectl create -f k8s/update/deployment.yaml # restart automatic updates (T289682)
  • 19:18 wm-bot: <lucaswerkmeister> moved all ZObject pages to main namespace using moveBatch.php (T289682)
  • 18:55 wm-bot: <lucaswerkmeister> kubectl delete deployment update # temporarily disable automatic updates since I’ll probably have to hand-edit the code for T289682

2021-08-22

  • 17:45 wm-bot: <lucaswerkmeister> namespace-protected the main namespace so only sysops can edit it, since ZObjects are expected to move into this namespace

2021-08-20

  • 19:55 wm-bot: <lucaswerkmeister> ran "php extensions/WikiLambda/maintenance/reloadBuiltinData.php" at ArthurPSmith’s request
  • 19:46 wm-bot: <lucaswerkmeister> ran "git -C extensions/WikiLambda/ submodule update --init --recursive", apparently the function-schemata submodule was missing?

2021-08-14

  • 19:21 wm-bot: <lucaswerkmeister> installed TemplateStyles extension (turns out it doesn’t do what I wanted to but let’s keep it anyways)

2021-07-26

  • 18:49 wm-bot: <lucaswerkmeister> installed MinervaNeue and MobileFrontend a few hours ago, forgot to log it earlier (T287401)

2021-07-15

  • 21:05 wm-bot: <lucaswerkmeister> updated pygments-server to 2d4ab81a68 (prevent open(); T286724)
  • 18:49 wm-bot: <lucaswerkmeister> block external traffic to internal pods (T286724)
  • 18:24 wm-bot: <lucaswerkmeister> don’t mount service tokens into pods (T286724)
  • 17:53 wm-bot: <lucaswerkmeister> only mount ~/www/python into pygments-server pod (T286724)

2021-06-21

  • 18:05 wm-bot: <lucaswerkmeister> fixed function orchestrator and evaluator

2021-06-05

  • 12:23 wm-bot: <lucaswerkmeister> wiki no longer read-only (T284368)
  • 12:11 wm-bot: <lucaswerkmeister> ran extensions/WikiLambda/maintenance/reloadBuiltinData.php, most objects seemed to work, only a few errors (T284368)
  • 11:53 wm-bot: <lucaswerkmeister> tried to run extensions/WikiLambda/development/updateLanguageCodes.php, most objects failed with the same “value en for Z11K1 … invalid” error as quoted in T284368
  • 11:36 wm-bot: <lucaswerkmeister> made wiki read-only for maintenance (T284368)

2021-06-02

  • 18:34 wm-bot: <lucaswerkmeister> deployed workaround for broken login (T284170#7129989)

2021-06-01

  • 19:38 wm-bot: <lucaswerkmeister> deployed function-evaluator as another service (only Node supported, not Python)

2021-05-29

  • 14:57 lucaswerkmeister: lowered CPU requests/limits of non-webservice deployments (cf. T283970#7123072)
  • 14:56 lucaswerkmeister: deployed function-orchestrator as another service
  • 11:12 lucaswerkmeister: moved kubernetes files into k8s/ and started tracking them in git

2021-05-28

  • 21:05 wm-bot: <lucaswerkmeister> deployed https://github.com/lucaswerkmeister/pygments-server rather than yesterday’s uncommitted version of it (most significant change: now on port 7879)
  • 00:17 wm-bot: <lucaswerkmeister> set up SyntaxHighlighting earlier this evening, with custom pygments-server as a separate kubernetes service

2021-05-27

  • 19:42 majavah: bump quota to 3 services T283754

2021-04-14

  • 21:40 wm-bot: <lucaswerkmeister> switched WSOAuth to Gerrit version

2021-04-10

  • 14:55 wm-bot: <lucaswerkmeister> installed Scribunto, WikiEditor and CodeEditor a few hours ago; all of them seem to work fine

2020-10-13

  • 14:28 wm-bot: <lucaswerkmeister> bumped update deployment to 3 GiB RAM

2020-10-04

  • 22:30 wm-bot: <lucaswerkmeister> php maintenance/createAndPromote.php --force --interface-admin --sysop --bureaucrat "Lucas Werkmeister"

2020-09-30

  • 23:40 wm-bot: <lucaswerkmeister> migrated `cron` from jstart to k8s, see deployment.yaml
  • 22:14 wm-bot: <lucaswerkmeister> installed cldr because Jforrester said so :)

2020-09-27

  • 15:48 wm-bot: <lucaswerkmeister> set $wgMainCacheType to CACHE_DB to hopefully make login sessions more stable