Help talk:Toolforge/Jobs framework/Archives/2023

Please do not post any new comments on this page. This is a discussion archive See current discussion or the archives index.

Cron syntax

Is it full cron syntax or just what is stated here? I mean 0 8 ? * MON#2 for every second Monday of the month, see

• https://stackoverflow.com/questions/49812967/cron-expression-for-every-second-monday-of-the-month-for-hangfire

binbot (talk) 08:30, 12 February 2023 (UTC)

@Bináris The Kubernetes specification doesn't explicitely specify which cron syntax is allowed, but looking at the source code it looks like it's using the standard mode of github.com/robfig/cron/v3, which corresponds with what's listed as 'standard' on the Wikipedia page.

The StackOverflow link you posted seems to be for the 'Quartz' syntax, which is not supported. You might find this more helpful: https://stackoverflow.com/questions/11683387/run-every-2nd-and-4th-saturday-of-the-month. Majavah (talk!) 10:14, 12 February 2023 (UTC)

@Majavah This looks sophisticated. :-) Great, thank you very much for your effort to look inside! binbot (talk) 14:00, 14 February 2023 (UTC)

PHP syntax

Hello. This page suggests the following syntax for PHP tasks (albeit when talking about memory allocations): "toolforge-jobs run myjob --command ./i_like_more_ram.php --image php7.4 --mem 1Gi --cpu 2". However, that doesn't work for me -- it seems like it ought to be "toolforge-jobs run myjob --command "php ./i_like_more_ram.php" --image php7.4 --mem 1Gi --cpu 2". Can someone with more understanding of the right syntax confirm? jarry1250 (talk) 12:20, 25 March 2023 (UTC)

@Jarry1250 I've updated the documentation. The original format worked if you'd marked the php script as executable and added a shebang, but the php script.php syntax is more common and beginner-friendly. Majavah (talk!) 12:52, 25 March 2023 (UTC)

Gotcha, thanks. jarry1250 (talk) 13:49, 26 March 2023 (UTC)

"Cron" needs better docs

Ok, so the Grid Engine way for periodic jobs was to create a crontab entry (bog standard *nix method) where you run either jsub or jlocal to create a Grid Engine job (or non-job in the case of jlocal). So in an effort to finally start transitioning my tool to Kubernetes I pick one of the cron jobs and change it to call toolforge-jobs instead of jsub. Which results in some clever function changing…

28 4 * * * toolforge-jobs run jobname [opts] --command "command" --emails all

…into…

28 4 * * * /usr/bin/jsub -N cron-49 -once -quiet toolforge-jobs run jobname [opts] --command "command" --emails all

Sigh.

So, clearly, what Help:Toolforge/Jobs_framework#Creating_scheduled_jobs_(cron_jobs) is trying to communicate is that in k8s-land we're not actually supposed to use cron, in favour of a built-in cron-alike facility in either the Toolforge Jobs Framework or in k8s. This seems rather surprising to me, since *nix cron is an incredibly well-established and well known and understood facility, but I'm guessing from a k8s perspective it probably enables some better orchestration or something.

In any case, this needs some actual explanation on Help:Toolforge/Jobs framework. For example a #Cron section that explains the difference, its advantages (the reasoning, primarily, to help understanding), documents the auto-conversion of crontabs if you try it the old way, and gives a couple of examples of old and new ways to run a periodic job.

In particular, it needs to say straight up that toolforge-jobs … --schedule "timespec" is the new One True Way™ if that is the case so people like me don't have to wonder whether we're messing something up and why it's not working.

And if it's not the One True Way™ but merely the old jsub-ification automagic for crontab that hasn't been removed yet, and going forward it should be possible to use cron too, as an alternative for us dinosaurs, then that ought to be mentioned too.

Oh, and "cron" is what you manipulate with crontab. If it's not managed through crontab and executed by crond then the docs shouldn't refer to it as "cron". Use "scheduled jobs" or something as the terminology to cut down on the confusion. --Xover (talk) 08:45, 6 April 2023 (UTC)

Oh, and the suggested docs might beneficially also refer to Help:Toolforge/Jobs framework#Loading jobs from a YAML file as a way to replicate some of the desirable properties of cron jobs. In particular, a crontab serves as both a configuration file for your jobs and an overview of what jobs are (intended to be) running. If you just interactively schedule something with toolforge-jobs … --schedule it'll be lost after a reboot or other such interrupt. Having inherited a large Toolforge tool from an inactive maintainer I can say with some empiric authority that the crontab was invaluable for figuring out what the heck was going on. --Xover (talk) 08:55, 6 April 2023 (UTC)

Job time limits?

Is is possible to set a time limit for a job? After migrating my jobs to Kubernetes I occasionally found that a job may got stuck for several days which I had to manually restart.This does not seem to happen before in the Grid Engine. Wcam (talk) 21:34, 4 May 2023 (UTC)

ERROR: Please report this issue to the Toolforge admins

Tracked in Phabricator
Task T340829 Resolved

for what it's worth

posting this here of the off chance it might possibly be useful to someone. Wbm1058 (talk) 21:04, 25 June 2023 (UTC)

---

tools.billsbots@tools-sgebastion-10:~$ toolforge-jobs restart refreshlinks
ERROR: An internal error occured while executing this command.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 71, in _make_request
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url:             https://api.svc.tools.eqiad1.wikimedia.cloud:30003/jobs/api/v1/restart/refreshlinks

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 712, in main
    run_subcommand(args=args, api=api)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 659, in run_subcommand
    op_restart(api, args.name)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 586, in op_restart
    api.post(f"/restart/{name}")
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 95, in post
    return self._make_request("POST", url, **kwargs).json()
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 75, in _make_request
    raise self.exception_handler(e)
tjf_cli.api.TjfCliHttpError: Internal Server Error
ERROR: Please report this issue to the Toolforge admins: https://w.wiki/6Zuu
tools.billsbots@tools-sgebastion-10:~$

-- Wbm1058 (talk) 21:04, 25 June 2023 (UTC)

Some background for the above. I have a "continuous" job running. I confirmed that it's still running with the "toolforge jobs list" command. Its log file has grown to 218,145 KB making this file too large to allow reading it via WinSCP. Previously when this log file became too large I've forced the creation of a new log with "toolforge jobs restart refreshlinks" but now this is failing, making it impossible to kill or restart my existing job which I suppose means it will just keep on running until it crashes your system. Wbm1058 (talk) 21:25, 25 June 2023 (UTC)

Now it's grown to 262,000 KB.

Still telling me to:

ERROR: Please report this issue to the Toolforge admins: https://w.wiki/6Zuu

Anybody home? Wbm1058 (talk) 17:47, 29 June 2023 (UTC)

Working for me again, thanks. Wbm1058 (talk) 19:37, 30 June 2023 (UTC)

Another error report for the Toolforge admins

Tracked in Phabricator
Task T340829 Resolved

Console dump:

tools-sgebastion-10:~$ toolforge jobs list
ERROR: An internal error occured while executing this command.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 57, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.7/socket.py", line 748, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 841, in _validate_conn
    conn.connect()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 301, in connect
    conn = self._new_conn()
  File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f0e4a9d4a58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.svc.tools.eqiad1.wikimedia.cloud', port=30003): Max retries exceeded with url: /jobs/api/v1/list/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f0e4a9d4a58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 712, in main
    run_subcommand(args=args, api=api)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 653, in run_subcommand
    op_list(api, output_format)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 328, in op_list
    list = _list_jobs(api)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 324, in _list_jobs
    return api.get("/list/")
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 124, in get
    return self._make_request("GET", url, **kwargs).json()
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 103, in _make_request
    response = self.session.request(method, **self.make_kwargs(url, **kwargs))
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 535, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 648, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.svc.tools.eqiad1.wikimedia.cloud', port=30003): Max retries exceeded with url: /jobs/api/v1/list/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f0e4a9d4a58>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
ERROR: Please report this issue to the Toolforge admins: https://w.wiki/6Zuu
tools.billsbots@tools-sgebastion-10:~$

Wbm1058 (talk) 13:07, 11 September 2023 (UTC)

Help moving pywikibot jobs including user-fixes.py customization

I'm trying to migrate my job in Kubernetes.

Currently, jsub scriptname.sh works fine.

I tried $ toolforge jobs run myjob --command ./scriptname.sh --image bullseye --emails all but my job fails with the message: ./scriptname.sh: line 3: python3: command not found

My script has only 3 lines:

#!/bin/bash

source $HOME/.bash_profile

python3 /data/project/shared/pywikibot/stable/scripts/replace.py -log -ns:0 -transcludes:Moduli:Citation/CS1 -fix:Pastrim -fix:Gjuhë -fix:Data fix:Data-riparim -fix:Përditësime -fix:Shtesa -fix:Padukshëm -fix:Dorazi -fix:Referime -fix:Rikthim -fix:Bosh -fix:NIA -fix:Prova -always

I'm stuck here. I thought I'd ask publicly for help so maybe other people can get help for similar situations as well. Any guidance would be appreciated. - Klein Muçi (talk) 02:56, 18 December 2023 (UTC)

@Klein Muçi It seems like you're using the bullseye image which only has the base Debian 11 Bullseye setup and is mostly intended for compiled languages. Try selecting a Python-specific image like python3.11. Majavah (talk!) 11:47, 18 December 2023 (UTC)

@Majavah, I tried using $ toolforge jobs run myjob --command ./scriptname.sh --image python3.11 --emails all and my job still fails with this traceback:

Traceback (most recent call last):
  File "/data/project/shared/pywikibot/stable/scripts/replace.py", line 155, in <module>
    import pywikibot
  File "/data/project/shared/pywikibot/stable/pywikibot/__init__.py", line 49, in <module>
    from pywikibot.bot import (
  File "/data/project/shared/pywikibot/stable/pywikibot/bot.py", line 112, in <module>
    from pywikibot import config, daemonize, i18n, version
  File "/data/project/shared/pywikibot/stable/pywikibot/version.py", line 26, in <module>
    from pywikibot.comms.http import fetch
  File "/data/project/shared/pywikibot/stable/pywikibot/comms/http.py", line 45, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

Klein Muçi (talk) 12:25, 18 December 2023 (UTC)

It sounds like you're trying to use a virtual environment created for the grid engine inside Kubernetes. You will need to re-create it following News/Toolforge_Grid_Engine_deprecation#venv, or if you're just trying to run scripts included in pywikibot, follow Help:Toolforge/Running Pywikibot scripts instead and avoid the venv setup hassle. Majavah (talk!) 12:28, 18 December 2023 (UTC)

My bot runs 2 kinds of jobs, one of which is just a large group of replacement lists (fixes), using the replace script in pywikibot. I'm currently only focusing on migrating that job so I'm following the second method. I set up a new OAuth 1.0a consumer for my bot. Updated my user-config.py file tokens to the new ones and set up the 5 envars. Then tried: $ toolforge jobs run myjob --command ./scriptname.sh --image python3.11 --emails all - The job still fails with the exact same traceback. - Klein Muçi (talk) 13:42, 18 December 2023 (UTC)

That is not the image and command format documented on Help:Toolforge/Running Pywikibot scripts. Majavah (talk!) 15:23, 18 December 2023 (UTC)

Tried:

$ toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb replace -log -ns:0 -transcludes:Moduli:Citation/CS1 -fix:Pastrim -fix:Gjuhë -fix:Data fix:Data-riparim -fix:Përditësime -fix:Shtesa -fix:Padukshëm -fix:Dorazi -fix:Referime -fix:Rikthim -fix:Bosh -fix:NIA -fix:Prova -always" myjob

Is that the correct way?

It still fails. I see no .err/.out files though so can't say why it fails. - Klein Muçi (talk) 16:53, 18 December 2023 (UTC)

This looks better, but seems like you did not specify which wiki to run the bot on. I've clarified the logging documentation a bit, unfortunately the image can currently only use the internal logging backend. Majavah (talk!) 17:24, 18 December 2023 (UTC)

That was intentional because I was using the one project and language I have set as my default on my user-config.py. Do I still need to add them?

I tried adding them and the job still fails. Using toolforge jobs logs I get:

family and mylang are not set.
Defaulting to family='wikipedia' and mylang='test'
ERROR: Incomplete command line pattern replacement pair.

Sometimes I also get this (thought it could be of interest)

ERROR: An internal error occured while executing this command.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 117, in _make_request
    response.raise_for_status()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 500 Server Error: INTERNAL SERVER ERROR for url: https://api.svc.tools.eqiad1.wikimedia.cloud:30003/jobs/api/v1/jobs/myjob/logs?follow=false

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/api.py", line 47, in handle_http_exception
    json = original.response.json()
  File "/usr/lib/python3/dist-packages/requests/models.py", line 897, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python3/dist-packages/simplejson/__init__.py", line 518, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/usr/lib/python3/dist-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 787, in main
    run_subcommand(args=args, api=api)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 722, in run_subcommand
    op_logs(api, args.name, args.follow, args.last)
  File "/usr/lib/python3/dist-packages/tjf_cli/cli.py", line 527, in op_logs
    params=params,
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 167, in get_raw_lines
    **kwargs,
  File "/usr/lib/python3/dist-packages/toolforge_weld/api_client.py", line 130, in _make_request
    raise self.exception_handler(e) from e
  File "/usr/lib/python3/dist-packages/tjf_cli/api.py", line 59, in handle_http_exception
    except requests.exceptions.InvalidJSONError:
AttributeError: module 'requests.exceptions' has no attribute 'InvalidJSONError'
ERROR: Please report this issue to the Toolforge admins: https://w.wiki/6Zuu

Klein Muçi (talk) 19:09, 18 December 2023 (UTC) - Klein Muçi (talk) 19:51, 18 December 2023 (UTC)

Adding some info here from related discussion on irc for posterity. Transcript has been lightly edited to remove other discussion threads that overlapped.

[09:53]  <    wm-bb> <Klein> Hey everyone! I'm still on my adventure trying to migrate my tasks in Kubernetes.
[09:53]  <    wm-bb> <Klein> 
[09:53]  <    wm-bb> <Klein> I'm using this command:
[09:53]  <    wm-bb> <Klein> 
[09:53]  <    wm-bb> <Klein> toolforge jobs run --image tool-pywikibot/pywikibot-scripts-stable:latest --command "pwb replace -log -ns:0 -transcludes:Moduli:Citation/CS1 -fix:Gjuhë -fix:Data -always"
[09:53]  <    wm-bb> <Klein> 
[09:53]  <    wm-bb> <Klein> But I get:
[09:53]  <    wm-bb> <Klein> 
[09:53]  <    wm-bb> <Klein> family and mylang are not set.
[09:53]  <    wm-bb> <Klein> Defaulting to family='wikipedia' and mylang='test'
[09:53]  <    wm-bb> <Klein> ERROR: Incomplete command line pattern replacement pair.
[09:54]  <    wm-bb> <Klein> 
[09:54]  <    wm-bb> <Klein> Which tells me my user-config.py and user-fixes.py files are not being considered at all when the job is running. I'm stuck at this point currently. :/
[15:51]  <    bd808> @Klein: The tool-pywikibot/pywikibot-scripts-stable:latest image does not mount and use files like user-config.py from your tool's $HOME. Instead it expects things like `-family:...` and `-lang:...` to be passed as arguments in the `--command` option to `toolforge jobs run`. See https://wikitech.wikimedia.org/wiki/Help:Toolforge/Running_Pywikibot_scripts for slightly more information.
[15:54]  <    bd808> If you need deeper customization, one possibility is forking https://gitlab.wikimedia.org/toolforge-repos/pywikibot-buildservice and adding your own customizations. I had to do this for a bot that I operate because its target wiki does not support OAuth authentication (<https://gitlab.wikimedia.org/toolforge-repos/officewikibot-pywikibot>).
[16:30]  <    wm-bb> <Klein> @bd808, I was following those suggestions (given by Taavi - thankful) when I reached the dead-end. I tried adding those family and language arguments (the error message should be changed to "lang" instead of "mylang" BTW) but I strangely get the same error message. I will try it again. Even if it works though, how do I manage to incorporate my fix files in the command if that ima <clipped message>
[16:30]  <    wm-bb> <Klein> ge doesn't use my files at all? No way at all to do that currently?
[16:34]  <    bd808> @Klein: At the moment there is no way to customize the code or config used by tool-pywikibot/pywikibot-scripts-stable:latest. The alternatives are to make your own custom build service image or to setup and run your scripts from a venv virtual environment maintained in your tool's $HOME using one of the pre-built python images.
[16:36]  <    bd808> We probably could use an "advanced tutorial" for doing that actually.
[16:40]  <   JJMC89> There used to be a pywikibot one for the venv method, but it was removed in favor of using the image or following the generic python venv one.
[16:48]  <    wm-bb> <Klein> Hmm... I tried some days ago reading the instructions in Wikitech about creating venvs but I was stuck when it was asking me to put what packages I wanted to install and I didn't know what to choose. I assume creating a custom image is even harder. Should I wait for a sort of guide? Or maybe someone can give me some information to read meanwhile and try to make my way in those mu <clipped message>
[16:48]  <    wm-bb> <Klein> rky waters? :P I'd take it slower but the deprecation deadline for the grid is close so...
[17:21]  <    bd808> JJMC89: hmmmm... yeah. Where was the prior information that you and others had worked on for that? The "If you want to run a script that is not included with Pywikibot itself, follow the generic Python on Toolforge documentation.' on taavi's new page leaves a lot to the imagination.
[17:22]  <   JJMC89> https://wikitech.wikimedia.org/wiki/Obsolete:Toolforge/Pywikibot
[17:26]  <    bd808> JJMC89: Thanks. I think we should bring that content back to the Help namespace. I can work on that later today if nobody beats me to it. I do think it should be done in a way that makes it clear that if you only want to run a default script the tool-pywikibot/pywikibot-scripts-stable:latest image should be the simplest path to success.
[17:30]  <    bd808> @Klein: https://wikitech.wikimedia.org/wiki/Obsolete:Toolforge/Pywikibot is probably the best documentation we have right now on how to do custom things with pywikibot.
[17:30]  <    bd808> @Klein: I would also say that it is ok to wait a bit longer for better help if 1) you make a note on your tool's grid shutdown task about what you are stuck waiting on help with and 2) there is some related phab task about fixing the thing that has you stuck.
[17:55]  <    wm-bb> <Klein> @bd808, thank you! I will try following the instructions in that page you just gave to me.
[17:55]  <    wm-bb> <Klein> 
[17:55]  <    wm-bb> <Klein> I did leave a comment on my tool's Phab task explaining my situation and a link to an ongoing discussion on a Wikitech talk page I was having with Taavi. But I didn't start any new tickets because it's not a single thing I'm stuck at. For example, in the beginning I was struggling with the basic concept of choosing the right image (was using bullseye only because that was the ima <clipped message>
[17:55]  <    wm-bb> <Klein> ge shown in the example on Wikitech). Also on many times I don't know if where I'm stuck is because of my own knowledge shortcoming or something is indeed missing somewhere and needs a Phab ticket. This is the first time ever I'm dealing with Kubernetes. It took me some months to understand what was going on in Grid Engine and not just use randomly suggested copypasta. I'll have  <clipped message>
[17:55]  <    wm-bb> <Klein> to go at that route again now with Kubernetes.
[17:57]  <    bd808> @Klein: :nod: For what it is worth, it sounds to me like you are doing all the right things. I also have empathy for needing time to learn new technology stacks beyond being able to cut-and-paste things.
[17:59]  <    wm-bb> <Klein> @bd808, thanks for the kinds words! :))
[18:01]  <    wm-bb> <Klein> https://wikitech.wikimedia.org/wiki/Help_talk:Toolforge/Jobs_framework#Job_migration_-_Help
[18:01]  <    wm-bb> <Klein> 
[18:01]  <    wm-bb> <Klein> Just to be exact, this is the link I referenced above. I was hoping to solve everything there to leave a kind of trail for people that might find themselves lost in the same woods. Anyway, if I manage to get through with this, I'll try to leave the pages there that ultimately helped me.

BryanDavis (talk) 18:16, 21 December 2023 (UTC)

The Obsolete:Toolforge/Pywikibot page mentioned in the above IRC discussion has now been moved to Help:Toolforge/Running Pywikibot scripts (advanced) and is linked to from Help:Toolforge/Running Pywikibot scripts. -- BryanDavis (talk) 23:14, 21 December 2023 (UTC)

@Majavah, following your suggestion on IRC to create a command similar to the example one specified in Help:Toolforge/Running Pywikibot scripts (advanced)#Create jobs, I created

toolforge jobs run myjob --command "$HOME/pwbvenv/bin/pwb replace -log -ns:0 -transcludes:Moduli:Citation/CS1 -fix:Pastrim -fix:Gjuhë -fix:Data -fix:Data-riparim -fix:Përditësime -fix:Shtesa -fix:Padukshëm -fix:Dorazi -fix:Referime -fix:Rikthim -fix:Bosh -fix:NIA -fix:Prova -always" --image python3.11

The job fails with this message:

ERROR: replace.py not found! Misspelling?

- Klein Muçi (talk) 16:30, 22 December 2023 (UTC)

After @JJMC89's edits on the Help:Toolforge/Running Pywikibot scripts (advanced)#Create jobs's page, I tried:

toolforge jobs run myjob --command "$HOME/pwbvenv/bin/python3 $HOME/pywikibot-core/pwb.py replace -log -ns:0 -transcludes:Moduli:Citation/CS1 -fix:Gjuhë -always" --image python3.11"

And it works!

The only problem is that currently I can't put multiple fix files else the job fails with the message:

ERROR: Incomplete command line pattern replacement pair.

I believe I need to find a way to put the list of fixes so that they don't confuse the interpreter. - Klein Muçi (talk) 11:46, 25 December 2023 (UTC)

Apparently the problem was coming from -fix:Data-riparim which included a hyphen and that broke the command. Now it works nicely altogether. Thank you to everyone involved for the walkthrough! - Klein Muçi (talk) 12:09, 25 December 2023 (UTC)