News/2020 PAWS migration
In order to encourage volunteer participation in and to improve WMCS support of the PAWS project, we have been rebuilding it in its own project with quite a number of design and deployment changes. For most users, the change should be totally transparent except for moving to a new domain name (which will have redirects), better service availability and upgrades to Jupyterhub.
When the transition is complete, we will be merging https://github.com/toolforge/paws/pull/51 into the main PAWS repo and the new canonical URLs for PAWS will be:
- https://paws.wmcloud.org (for the wiki page)
- https://hub.paws.wmcloud.org (for PAWS hub itself)
- https://public.paws.wmcloud.org (for paws-public links)
- paws.wmflabs.org/hub will redirect to hub.paws.wmcloud.org and paws-public.wmflabs.org will redirect to public.paws.wmcloud.org
What is changing?
- New domain names for the PAWS hub and paws-public
- The underlying Kubernetes cluster is getting a big upgrade and will be much more reproducible with WMCS monitoring in place. It is built more like the Toolforge Kubernetes cluster with changes to be more appropriate to Jupyterhub hosting.
- Moving to Jupyterhub version 1.1.0
- All images were rebuilt with some upgrades or corrections
- "paws-public" will no longer be hosted in Toolforge. Rather it will be hosted on the same cluster as PAWS.
- Dumps NFS will fail over correctly and thus be more reliably available to notebooks
- PAWS is moving to its own Cloud VPS project with distinct permissions and NFS from Toolforge. The existing NFS will be synced over.
Timeline
- 2020-07-31: Initial announcement after fully preparing a working cluster in parallel for the community to start trying out.
- 2020-08-07: Presuming any bugs found in the preceding weeks are fixed, merge the Pull Request, do a final NFS sync and change DNS to redirect to the new cluster. The old cluster will be shut down but not totally cleaned up.
- 2020-08-14: Clean up the old cluster.
What should I do?
Please try out the new cluster
The new cluster can be accessed at https://hub.paws.wmcloud.org/. An initial NFS sync has been done, so a lot of your work will already be present for you to try out. New work done on the new cluster will remain in place, but do not create new files on the cluster with the same names as the ones that you only have on the old one. The final sync could overwrite them! If you find any bugs in the new cluster please create a task tagged with #PAWS and feel free to ping bstorm on the task!
Make sure things that work on the old cluster also work in the new one. We are hoping to replicate all functionality.
Why are we doing this?
The original cluster suffers from support neglect, single points of failure and limited reproducibility. It was in bad need of documentation, support, upgrades and bug fixes. Importantly, the changes being made are aimed at making the project more inclusive and open to contribution and further improvements in the future.
In addition to those goals, this change removes another service behind a "labs" domain name, moving it to a supported "cloud" one.
After the public testing and migration
After the final switch over of DNS, the paws.wmflabs.org and paws-public.wmflabs.org domain names will be redirects that should allow existing paws-public material to continue to work correctly on the web. Otherwise, we hope that PAWS will simply be a more stable and well-managed environment with better opportunities for people to contribute to and extend the project.