Help talk:Toolforge/My first Flask OAuth tool

This talk page is not a Cloud VPS or Toolforge support forum. Contact information for Cloud VPS and Toolforge can be found at Help:Cloud Services communication.

pip

Of what host is this talking about?

tools.bub@tools-bastion-01:~/public_html/BUB/bot$ pip install --user internetarchive
The program 'pip' is currently not installed. To run 'pip' please ask your administrator to install the package 'python-pip'

--Nemo 11:37, 1 June 2015 (UTC)

Nemo bis, is pip still not installed? That sounds like a bug in the environment. Alternatively is the command virtualenv installed? John Vandenberg (talk) 21:04, 21 December 2015 (UTC)

The pip program is purposefully not installed globally on the bastions or other Tool Labs servers. In the tutorial, use of pip ... commands assumes that you have already activated the virtual environment for your shell session by running source $HOME/www/python/venv/bin/activate. Sourcing that script alters your shell sessions' $PATH so that $HOME/www/python/venv/bin/pip is in the default search path. --BryanDavis (talk) 05:21, 6 February 2017 (UTC)

"Address already in use"

I am getting "socket.error: [Errno 98] Address already in use". I have tried "webservice2 uwsgi-python stop" and "webservice2 uwsgi-python restart" as well as "webservice uwsgi-python stop". Nothing seems to help. With "ps -ef | grep scholia" I see three relevant lines. These processes are owned by root. Here is a few messages from

   (venv)tools.scholia@tools-bastion-03:~$ webservice2 uwsgi-python start
   Your job is already running
   (venv)tools.scholia@tools-bastion-03:~$ webservice2 status
   Your webservice is not running
   (venv)tools.scholia@tools-bastion-03:~$ webservice2 uwsgi-python restart
   Restarting webservice...............
   (venv)tools.scholia@tools-bastion-03:~$ webservice2 status
   Your webservice is not running

It seems that I got started processes I cannot stop. — Finn Årup Nielsen (fnielsen) (talk) 17:04, 10 October 2016 (UTC)

For readers from the future (hi!) we worked through this on irc. The hanging jobs were in shutdown state but not exiting so they were force killed using sudo qdel $JOBID. The "Address already in use" error seemed to be only semi-related and instead caused by incorrect PYTHONPATH settings which led to the wsgi process crashing. --BryanDavis (talk) 17:43, 12 October 2016 (UTC)

Major rewrite of content

I have boldy rewritten much of the content and moved the page from Help:Tool Labs/Python application stub to Help:Tool Labs/My first Flask OAuth tool. Some more things I would like to see added are listed in task T134494. --BryanDavis (talk) 05:16, 6 February 2017 (UTC)

Making the tool HTTPS only

Using toolforge helper library:

from flask import Flask
import toolforge

app = Flask(__name__)
app.before_request(toolforge.redirect_to_https)

Is this beginner enough to mention in the tutorial? Or should it just be a footnote? legoktm (talk) 07:54, 26 August 2017 (UTC)

I think either would be fine, but it might be easiest to add in as a "what next" step. That could be as easy as linking to User:Legoktm/toolforge library. --BryanDavis (talk) 17:51, 26 August 2017 (UTC)

Done I added Bryan’s suggestion for now. --Lucas Werkmeister (WMDE) (talk) 09:29, 19 June 2018 (UTC)

Very useful

As a Flask and Toolforge newbie this guide was extremely helpful to getting set up. Thanks to all who wrote it (from the history, Little green rosetta, Merlijn van Deen, and BryanDavis)! Samwalton9 (talk) 14:43, 10 November 2017 (UTC)

virtualenv not present in kubernetes container

@APerson:, I feel that edit that you made about using virtualenv is misleading. There is no virtualenv script installed inside the Kubernetes python container. I think that it is possible that the command you provided will work from a Toolforge bastion, but even then the generated venv may be subtlety incompatible with the version of Python that is used in the Kubernetes containers.

$ webservice --backend=kubernetes python shell
$ virtualenv $HOME/www/python/venv -p /usr/bin/python3
bash: virtualenv: command not found

I did not want to revert without some discussion here however in case there are other issues that I have missed. Maybe the instruction to perform the commands from within a Kubernetes managed container are not prominent enough? --BryanDavis (talk) 21:47, 7 October 2018 (UTC)

BryanDavis, oh, I see. Yes, I evidently missed the first command. I was going off this discussion on the WMF Phabricator where the command I used was recommended (I found it by searching for the error message I got when trying the normal command). I have amended the section to indicate that you should only get the error message I got if you forgot to run the first command. I'm also fine with removing the paragraph all together if it might confuse users. Thanks for pointing this out! APerson (talk!) 05:31, 8 October 2018 (UTC)

Developer mode

Hi, is there a possibility to use the advantages of the developer mode on Toolforge? It would be helpful, if Flask would recognize changed files. At the moment, I have to restart the webservice again and again to reflect these changes. Any ideas? --FNDE (talk) 20:40, 14 July 2019 (UTC)

If "developer mode" means https://flask.palletsprojects.com/en/1.1.x/server/, there is not an easy way to do that today with the webservice command. It should be possible to accomplish the flask run method using a custom deployment as described in Help:Toolforge/Kubernetes#Kubernetes_continuous_jobs, but that would be a very advanced thing to add to a "my first X" tutorial. Adding a touch-reload option to a $HOME/www/python/uwsgi.ini config file and then using touch the-path-you-configured to trigger a reload might get close to doing what you want, but I have not tried that personally. See Help:Toolforge/Web#Default_uwsgi_configuration for a bit more info on $HOME/www/python/uwsgi.ini. --BryanDavis (talk) 00:17, 15 July 2019 (UTC)

OAUTH_MWURI

What is app.config['OAUTH_MWURI'] in the last code? How can I find it? Also when I register of a new OAuth consumer, it says You have been assigned a consumer token of aaaaaaaaaaaaaaaaaaa and a secret token of bbbbbbbbbbbbbb Please record these for future reference. Am I getting it right that in config.yaml it should be

• CONSUMER_KEY: aaaaaaaaaaaaaaaaaaa
• CONSUMER_SECRET: bbbbbbbbbbbbbb

? Thank you very much!--Ashot1997 (talk) 16:32, 27 June 2020 (UTC)

The OAUTH_MWURI setting seems to have disappeared from the tutorial body, but it is documented in the related git repository. The default value should be something like "OAUTH_MWURI: https://meta.wikimedia.org/w/index.php".

Your mapping of the MediaWiki OAuth terminology to the tutorial's terminology is correct. --BryanDavis (talk) 16:14, 29 June 2020 (UTC)

Hi, BryanDavis. Thank you very much for your answer ^_^.--Ashot1997 (talk) 04:47, 2 July 2020 (UTC)

Installed packages in env report ModuleNotFoundError

After installing packages by `running pip install -r path-to-requirements.ext`, it completes cleanly. However, restarting the webservice reports that these modules are not found causing the application to fail with an internal server error. This is something I have been doing long before now but it seems to fail all the time. Am i missing something this time around?

PS: The said tool is the `isa-dev` tool on which we want to test new features and bug fixes before deploying to the live tool `isa` Eugene233 (talk) 01:10, 12 February 2024 (UTC)

@Eugene233 Your virtual environment was created for Python 3.11 but you're trying to run the web service on Python 3.9. Try completely stopping the web service, updating service.template for 3.11, and then starting it. Taavi (talk!) 08:14, 12 February 2024 (UTC)

YAML -> TOML conversion

@Gouvernathor, your series of edits converting the tutorial's config file from YAML to TOML have caused the tutorial to significantly differ from the published tool and its source. I'm not sure I understand the reasoning behind this change and was hoping you could expand a bit on the general value of using TOML over YAML that compelled you to invest the time in making these changes.

If we keep the TOML conversion then it would be good to see the tutorial tested and updated to fix any new bugs that a copy-and-paste deployment reveals. I'm pretty sure there is at least one in Help:Toolforge/My first Flask OAuth tool#How to add a configuration file where GREETING: Goodnight moon! is placed in a TOML file. I think it would also be reasonable to update the git repo with a merge request making equivalent changes to the final wiki edits.

Alternately, we can roll it back to the prior YAML implementation and someone can start a different tutorial that shows off TOML. -- BryanDavis (talk) 21:19, 21 May 2024 (UTC)

I don't know how to update the repo, but I can probably do it, I'm using a toml config file locally and it works very well.

The main advantage is that toml can be read using the standard library, whereas yaml requires a dependency. 3.11 is required, but it's the version the tutorial uses. Toml is also a simpler standard, more readable, and the parsing is most likely faster. Gouvernathor (talk) 17:01, 26 May 2024 (UTC)

> Toml is also a simpler standard^{[citation needed]}, more readable^{[citation needed]}, and the parsing is most likely faster^{[citation needed]}.

I very definitely have biases here as the maintainer of PHP's libYAML bindings, but it also seems your arguments can be reduced to personal preference at this point. That's a fine thing; we all have preferences. I would appreciate however the tutorial being worked on until it is possible to cut and paste all changes from the wiki into a test deployment without error again and to have the git repo updated with working changes to match. Folks are rightly frustrated when tutorial content is not functional. -- BryanDavis (talk) 16:17, 28 May 2024 (UTC)

> more readable

https://toml.io/en/

> simpler standard

Just compare the TOML full spec vs YAML's in terms of size. In terms of contents it's even clearer.

> parsing is most likely faster

The TOML parsing library is part of the standard library, which holds itself to stricts standards of performance. You can also compare the length of the code which is a not-at-all strict but indicative metric of what is simplest at execution : TOML's entire parsing library holds in 820 lines of code whereas YAML's is over 1600.

I would definitely hear the argument that YAML can be more concise and offer more freedom to those who write in it (I wouldn't want to have to write runners in TOML), but I honestly don't think the readability, and technical simplicity can be challenged.

As for the tutorial : I can write the code, I just have no idea how to write on the repo. Maybe I can just fork it on GitHub or something, then you can pull what I commit, switch remotes, and push it on the repo ? I'll do that when I find the time.

But what I've seen of the code is quite outdated in terms of python by certain aspects, so I may go further than that in terms of changes. You'll keep what you want. Gouvernathor (talk) 21:10, 28 May 2024 (UTC)