Help talk:Toolforge/Running Pywikibot scripts (advanced)/Archives/2024

Please do not post any new comments on this page. This is a discussion archive See current discussion or the archives index.

Tutorial doesn't work

I followed this tutorial part by part, up until step 5. If i try to run pwb, I get the following:

(pwbvenv) tools.eswstool2@tools-sgebastion-10:~$ pwb
Traceback (most recent call last):</code>
  File "/data/project/eswstool2/pwbvenv/bin/pwb", line 5, in <module>
    from pywikibot.scripts.wrapper import run
ModuleNotFoundError: No module named 'pywikibot'

It's a completely fresh tool. no .bash_profile in my files Ignacio Rodríguez (talk) 19:48, 13 March 2024 (UTC)

The shell prompt (pwbvenv) tools.eswstool2@tools-sgebastion-10:~$ strongly implies that you forgot to enter a Kubernetes container via webservice python3.11 shell somewhere along the way. The only things that should happen directly on the bastion are interactions with toolforge jobs ... and webservice python3.11 shell.

The specific error message ModuleNotFoundError: No module named 'pywikibot' means that the python venv that you are using ("/data/project/eswstool2/pwbvenv" per the prior trace line) does not have the pywikibot package installed inside of it. That should have happened via the tutorial's "5. Run the setup script using the jobs framework." instruction. As you are reporting that you were on step 5, but you are also showing the output of something different than toolforge jobs run setup-venv --command $HOME/pwb_venv.sh --image python3.11 --wait it is difficult to say where things went wrong with confidence. -- BryanDavis (talk) 20:09, 13 March 2024 (UTC)

Can confirm I forgot to enter a Kubernetes container and that was the whole issue.

Still don't know why my bot randomly stopped working 3 days ago, that led me to me and madness. Ignacio Rodríguez (talk) 21:02, 13 March 2024 (UTC)

New dependencies

Something change a few days ago, not all dependencies were met in the container (missing packaging, but probably others too). I had to manually set it up to run python3 -m pip install -r requirements.txt from the pywikibot folder. strainu (talk) 21:24, 13 March 2024 (UTC)

I expect you are modifying PYTHONPATH like in #deprecation/ModuleNotFoundError above. — JJMC89 (T·C) 21:33, 13 March 2024 (UTC)

Yes. Any reason I shouldn't be using the shared files? strainu (talk) 21:52, 13 March 2024 (UTC)

They are (intentionally) not mentioned anywhere on this help page. While you can use them, you do so at your own risk of issues like the one you just encountered (because they are automatically updated) and shouldn't be posting here about issues you encounter while doing so as they are out of scope. — JJMC89 (T·C) 22:12, 13 March 2024 (UTC)

I'm not sure why you assumed I was reporting an issue, my intention was to answer Ignacio Rodríguez in the section above. Nonetheless, if you put it this way, offering a set of tools without support seems like a bad idea to me. strainu (talk) 22:23, 13 March 2024 (UTC)

They're still there because the Grid Engine is still running. Once it shuts down, they could be removed. — JJMC89 (T·C) 22:34, 13 March 2024 (UTC)

deprecation/ModuleNotFoundError

Hello. My replace.py script was running without any changes since last 7-8 months. A couple of days ago, only one run of the script has been affected (I run it twice with different parameters). When I tried to run it for experiment, I got this message:

@tools-sgebastion-11:~$ toolforge-jobs load cronjobs.yaml
/usr/bin/toolforge-jobs:15: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources import load_entry_point

and in the .err file, there was following content:

Traceback (most recent call last):
  File "/data/project/kiranbot4/pywikibot-core/scripts/replace.py", line 155, in <module>
    import pywikibot
  File "/data/project/shared/pywikibot/stable/pywikibot/__init__.py", line 21, in <module>
    from pywikibot import config as _config
  File "/data/project/shared/pywikibot/stable/pywikibot/config.py", line 60, in <module>
    from pywikibot.backports import (
  File "/data/project/shared/pywikibot/stable/pywikibot/backports.py", line 210, in <module>
    from pywikibot.tools import ModuleDeprecationWrapper  # noqa: E402
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/project/shared/pywikibot/stable/pywikibot/tools/__init__.py", line 41, in <module>
    import packaging.version
ModuleNotFoundError: No module named 'packaging'

@JJMC89: Any idea how can I get the script working again? Thanks a lot in advance. courtesy ping @Klein Muçi, BryanDavis: —usernamekiran (talk) 16:54, 11 March 2024 (UTC)

Remove the PYTHONPATH modification you have in ~/.bash_profile so you are not using the shared pywikbot files (/data/project/shared/pywikibot). — JJMC89 (T·C) 17:51, 11 March 2024 (UTC)

@JJMC89: thanks, it worked. but I am still getting the notice about pkg_resources. should I update something so the pywikibot would not stop working in the future, or would it be done by the WMF? Regards, —usernamekiran (talk) 11:57, 12 March 2024 (UTC)

For toolforge-jobs? That's maintained by the WMCS team. It may go away after the bastions are updated to a new OS version (newer python version) following the Grid Engine removal. — JJMC89 (T·C) 15:52, 12 March 2024 (UTC)

I get the same warning. I believe it will be fixed without any action required from us as "end users". - Klein Muçi (talk) 17:40, 22 March 2024 (UTC)

No module named

Hello I'm tying to run job but this error No module named 'packaging' is showing I have already dependency installed I also tried to modify .bash_profile with command "nano ~/.bash_profile" but nothing found in this please help me to solve this issue Kaleem Bhatti (talk) 06:43, 17 March 2024 (UTC)

@Kaleem Bhatti Which tool account is this on? What exact command are you using to run the job? Taavi (talk!) 11:55, 17 March 2024 (UTC)

@Majavah I'm using job toolforge jobs run cats --command "$HOME/cats.sh" --image python3.11 --schedule "* * * * *" and in cats.sh file with code

python3 -m venv pwbvenv
$HOME/pwbvenv/bin/python3
$HOME/categorize/pwb.py categorize.py -recentchanges:500 -always -lang:sd /dev/null

and tool name is kaleem-bot Kaleem Bhatti (talk) 12:29, 17 March 2024 (UTC)

There's nothing installed in the virtual environment. (See your setup-venv.err.) cats.sh should just have $HOME/pwbvenv/bin/python3 $HOME/categorize/pwb.py categorize.py -recentchanges:500 -always -lang:sd (one line) and not be creating an empty venv. — JJMC89 (T·C) 16:24, 17 March 2024 (UTC)

I have tried this but still showing same error Kaleem Bhatti (talk) 17:04, 17 March 2024 (UTC)

There's still nothing installed in the virtual environment. — JJMC89 (T·C) 17:16, 17 March 2024 (UTC)

how I can install can you help me I have tried with pip install -r requirements.txt Kaleem Bhatti (talk) 17:22, 17 March 2024 (UTC)

Follow the instructions on the help page. — JJMC89 (T·C) 17:31, 17 March 2024 (UTC)

I have already followed but not working if you know any solution please help me I am disturbing a lot of Kaleem Bhatti (talk) 17:36, 17 March 2024 (UTC)

Please help me I have installed all dependency still showing No module named 'packaging' Kaleem Bhatti (talk) 07:34, 18 March 2024 (UTC)

@Xqt did you solution of this? or this is a bug Kaleem Bhatti (talk) 11:44, 18 March 2024 (UTC)