Toolforge workgroup meeting notes

Attendees

Andrew Bogott
Arturo Borrero
David Caro
Francesco Negri
Taavi Väänänen
Nicholas Skaggs
Raymond Olisaemeka

Agenda

Toolforge build service updates
- https://etherpad.wikimedia.org/p/second-round
Toolforge overall design discussion: task T342077
Toolforge as an open source PaaS

Notes

Toolforge build service updates:

DC: Build Service Updates. New email sent proposing second round of beta. Have envars and stabilized harbor. Should be much better experience for users. What does everyone think about moving forward with sending the email? Other ideas or suggestions?

DC: Mostly python/php tools targeted for the second beta.Going to start looking at grid tools as well, but not officially part of invite.

TV: We can continue, some minor nitpicks for messaging to share

FN: List of tools and admins? How are the two lists connected?

DC: Yes, list of tools and admins

NS: Are grid tools effectively unlocked? Barring push to deploy, major features are done?

DC: Yes. They might be using something we don’t support. For example, using two buildpacks at once. Availability of envvars are build time, would make it easier to script. Push to deploy (trigger a build, and deploying) both still need development.

NS: So grid isn’t part of beta2 but could be in the future

DC: Yes, want to try personally and share

Toolforge overall design discussion: task T342077

DC: Idea is to think about what toolforge can evolve into. For example, should it be a monolith, single client or many smaller clients, etc. Slavina intends to make a decision request eventually.

FN: What goal do you have in mind?

DC: Two reasons. Trying to validate what everyone thinks is the goal. Making sure everyone has the same standard to measure things. Reducing the complexity is a good goal.

FN: That’s valid. In a larger team, it can be a different problem. There is an effect. This will remove some freedom in submodules.

ABG: FN’s comment was excellent. Example of different teams having different components in toolforge. That’s exactly what Conway’s law would predict. In the situation shared by FN,. there’s inverse Conway’s law. Which is to model the team against the software you want to produce. Fascinating. Perhaps this is the right opportunity to find the right team practices to produce toolforge with the right technologies, etc.

DC: If software and team are organized differently, it can cause friction. Currently organized in strong individual contributors. This helps volunteers contribute easily. If we create a monolith, it will be harder to contribute. Have to follow rules, learn things, etc.

TV: Good thing to aim for all of toolforge components to aim like one big thing. It’s easier for the user.

ABG: Having lots of repos with smaller components, it may be more difficult to get the big picture. It may help to see everything in one git tree for new contributors.

DC: Two things. Monorepo and monolith. Monorepo, easier to grok, one repo. Monolith can be harder to reason about. Monolith means you need to understand everything to debug, can’t just reason about a single API.

FN: Liked the idea in general to avoid too many technologies. Should put some limits into how many languages that are allowed. It’s true having the monolith will be harder for new contributor. Need more time to understand everything that’s in play. For example, if you look only at toolforge jobs, could be faster to start since it’s small. Monolith isn’t that big inside toolforge. Don’t think the project is too big to understand.

RO: Right now, lots of repos. Can see them in gitlab. For installation, monorepo is easy. Don’t have to handle all the components differently. Today, have toolforge cli discovering other things that are installed. From a user perspective, it will be the same. But monorepo will simplify installation. On the code contribution side, tried to think about first time trying to push code. It was in gerrit, but there was mental overload. Because of so many things in different places. Can still be modular in the same codebase. Argument for monorepo makes sense. Monorepo for the backend ; unclear how that makes things easier.To clarify, speaking about the CLI. On the API side, still undecided.

DC: Not yet giving my opinion, trying to facilitate discussion. Bit of confusion between what user sees and how it’s implemented. Today, single API gateway, backed by many services. Or can have one single API.. User will see the same. Can be implemented as monolith or microservices. One repo or many. Don’t have to chose in a binary way. Can have big monorepo, or 3 “big” repositories. Can have a monolith, with some extra microservices. Consider the gray lines in between, not all black and white.

DC: Deployment strategies also play into this. Can split into failure domains, etc.

FN: Thinking about monolith vs monorepo. How can we separate them? Yes, we should separate conversations. Can we avoid many decision requests? Let’s make sure it's as productive as possible.

NS: Is the intention to narrow options now before decision requests?

DC: Not sure what Slavina is thinking. But wanted single API and single client. On how to make this productive, keep in mind what we are discussing. Don’t need to define the end state, think about what steps we can talk and are open to revising as we start walking in a specific direction. This will happen more in decision request.

Toolforge as an open source PAAS

FN: Same concerns we were discussing about volunteers. Open source project / open to wiki

TV: Probably not possible to replicate PaaS. But smaller components might make sense to build in a way others can use them as well.

DC: Personally like the idea. Re-use components, even part of it. Yes, there’s the contribution part of it, but also the reuse part. Having separate independent modules could allow you to take pieces of it. Any approach could still allow this. Some would make it easier. The entirety, others probably wouldn’t want to use.

FN: Might want to separate services / APIs, if we want components to be reusable, it will change design. Think about cut in line to integrate with platform or system.

DC: Separate by function?

FN: Need to think more about it. Having wiki specific things abstracted away.

NS: Buildpacks implementation might allow for reuse. It will inform design. But, we don’t have to do this!

Feedback about toolforge-deploy

DC: Has everyone tried it? thoughts?

TV: Definitely easier! Wait. components or tools?

DC: Components

DC: Reach out if you want to try and I’ll loop you in next time I deploy

FN: Think about how to make it work on cloudcumins, without global root. Could you ssh as a non-root?

DC: Should be able to. Just need toolforge admin. Want toolforge / non-global roots to do it.

FN: Something to keep in mind and test

TV: we've historically had plenty of toolforge roots without global/cloud-vps roots, and it'd be ideal if those could be able to run toolforge specific cookbooks on cloudcumin* hosts

Action items

dcaro to send toolforge build service beta 2nd round of invites after Taavi has reviewed the email