Talk:News/Toolforge Trusty deprecation

login-stretch.tools.wmflabs.org

Will the login-stretch.tools.wmflabs.org hostname stay up for the foreseeable future, even after the migration is completed? I. e. is it safe to put in my ssh_config, or should I stick to login.tools.wmflabs.org? --Lucas Werkmeister (talk) 23:53, 14 January 2019 (UTC)

We will keep it around and announce on the cloud-announce list with a reasonable deprecation period before it is removed. --BryanDavis (talk) 04:02, 15 January 2019 (UTC)

Thanks, sounds good :) --Lucas Werkmeister (talk) 21:07, 17 January 2019 (UTC)

lighttpd backend ?

Help:Toolforge/Web#Default_web_server_(lighttpd_+_PHP) and Help:Toolforge/Web/Lighttpd state that `lighttpd` is a backend for both gridengine and k18s. However, trying to run for one of my tool webservice --backend kubernetes lighttpd-plain start results in type must be one of golang,jdk8,nodejs,php5.6,php7.2,python,python2,ruby2,tcl. Switching to a php backend does not do the trick (I guess because this is a HTML based tool). Am I doing something wrong?

Jean-Frédéric (talk) 09:05, 8 February 2019 (UTC)

@Jean-Frédéric: The php7.2 backend on Kubernetes should work nearly identically to the lighttpd-plain backend on grid engine. At the lighttpd configuration level, the only difference is that the php7.2 backend includes a fastcgi server to handle .php files found in $HOME/public_html. The rest of the lighttpd configuration is identical. I think it would be worthwhile for you to open a Phabricator task reporting the problems you are seeing so that we can try to reproduce them and find a fix. --BryanDavis (talk) 17:48, 8 February 2019 (UTC)

How to verify move to Kubernetes

Since webservices moved from Trusty to Kubernetes will not be removed from the Trusty monitoring tool, is there a monitoring tool available to verify that webservices are running on Kubernetes? --Ahecht (talk) 15:01, 10 February 2019 (UTC)

There is not any publicly viewable list of things running on the Toolforge Kubernetes cluster at this time. I thought there was a more detailed ticket about this, but the only one I can find is phab:T201892. The primary issue is that we need to create an "observer" account that can inspect all of the Kubernetes namespaces to find running pods. Investigating this is currently waiting on our upgrade of the Kubernetes cluster itself (phab:T214513). --BryanDavis (talk) 23:26, 10 February 2019 (UTC)

tomcat webservice

I have three tools as Gradzeichen on trusty. They are tomcat webservices. If I try to run these on kubernates, it says tomcat is not supported. They run on trusty grid. If I start in stretch grid, the webservice is started, but if I try to access the website from the browser, it says no webserver is running. How do I go on? --G (talk) 19:10, 4 March 2019 (UTC)

From the logs of the isbn tool, it looks like it never succeeded in launching on the new grid. The java version on stretch doesn't match the one on Trusty, and I imagine you'll need to build the project on the new version to get it to run. There may be other concerns in that version upgrade as well. JDK 11 is running on the stretch cluster. --Bstorm (talk) 23:15, 4 March 2019 (UTC)

If I start the war-file on stretch, classes are loaded fine (with verbose option), the JVM then exits with no main class/no manifest entry (as would be expected, a war file is not meant to be started from the JVM, but from tomcat or another servlet implementation). So the war file can be interpreted by Java11. Are there other tomcat tools running on stretch? What version of tomcat and servlet are running on stretch (trusty: tomcat7, servlet3) --G (talk) 05:19, 6 March 2019 (UTC)

> Are there other tomcat tools running on stretch?

Not yet, no. There are only 8 tools in all of Toolforge running tomcat. None of them seem to have managed to move to the Stretch grid yet.

> What version of tomcat and servlet are running on stretch (trusty: tomcat7, servlet3)

tomcat 8.5.14, servlet 3.1

You might want to take a look at the discussion on phab:T141396 and consider if you can convert your tool to run in our Kubernetes jdk8 container instead of using the grid engine tomcat configuration. --BryanDavis (talk) 06:15, 6 March 2019 (UTC)

@BryanDavis: you raised the weblimit on Ireas request for the tool isbn, could you please do the same for isbn-tmptest and stereoskopie. I think, then I will be able to get both tools running on stretch. --G (talk) 20:25, 14 March 2019 (UTC)

@G: Done. Both now have a 6G limit. --BryanDavis (talk) 21:04, 14 March 2019 (UTC)

I may have been to optimistic:

gradzeichen@tools-sgebastion-07:~$ become isbn-tmptest
tools.isbn-tmptest@tools-sgebastion-07:~$ webservice status
Your webservice is not running
tools.isbn-tmptest@tools-sgebastion-07:~$ webservice tomcat start
Your job is already running
tools.isbn-tmptest@tools-sgebastion-07:~$ webservice stop
Your webservice is not running

tools.isbn-tmptest@tools-sgebastion-07:~$ webservice tomcat start
Your job is already running

--G (talk) 11:35, 19 March 2019 (UTC)

@G: That sounds exactly like News/Toolforge_Trusty_deprecation#'webservice_stop'_says_service_is_not_running,_but_'webservice_start'_says_service_is_running --BryanDavis (talk) 15:34, 19 March 2019 (UTC)

I deleted service.manifest, but it did not fix it.--G (talk) 20:00, 19 March 2019 (UTC)

$ sudo become isbn-tmptest
$ qstat-full
887558  0.27518  tomcat-isbn-tmptest  tools.isbn-tmptest  r  2019-03-16T14:13:58  webgrid-generic@tools-sgewebgrid-generic-0902.tools.eqiad.wmflabs  1
$ webservice status
Your webservice is not running
$ qdel 887558
tools.isbn-tmptest has registered the job 887558 for deletion
$ sleep 60
$ qstat-full
$ webservice status
Your webservice is not running
$ webservice tomcat start
Starting webservice............
$ webservice status
Your webservice of type tomcat is running

--BryanDavis (talk) 21:10, 19 March 2019 (UTC)

Python 3.4 is deprecated

Tracked in Phabricator
Task T219091

$ webservice --backend=kubernetes python shell
$ python3 -m venv www/python/venv/
$ source www/python/venv/bin/activate
$ pip install --upgrade pip
DEPRECATION: Python 3.4 support has been deprecated. pip 19.1 will be the last one supporting it. Please upgrade your Python as Python 3.4 won't be maintained after March 2019 (cf PEP 429).

Would be nice if a newer Python3 could be provided. – Danmichaelo (talk) 19:13, 23 March 2019 (UTC)

I am planning to roll out a Python3.5 Kubernetes image based on Debian Stretch soon. The code changes are prepared, but include features that I do not want to put into use until after the News/Toolforge Trusty deprecation is completed. --BryanDavis (talk) 21:57, 24 March 2019 (UTC)