User:DannyS712/sandbox

From #wikimedia-operations (publicly logged)
13:53 <DannyS712> dumb question: is there like a "Wikimedia infrastructure for dummies" that explains how puppet works / what restbase is / buster / everything?
13:53 <Majavah> actually a very good question
13:54 <Majavah> wikitech documentation is usually fairly technical, so not sure if we have a "for dummies" guide
13:54 <Pchelolo> I haven't seen one
13:55 <Majavah> Buster is the codename of Debian's version 10, Debian is the Linux distribution we use, that's the easiest to answer of those you mentioned
13:55 <mutante> DannyS712: that place is supposed to tbe the wikitech wiki
13:55 <Urbanecm> DannyS712: i would say there's not really one. Wikimedia infrastructure is REALLY complex
13:55 and I don't think there's ever going to be a "tldr" version of it
13:56 <legoktm> it's a wiki, when you run into something you don't know, ask, and then document it :)
13:57 <apergos> there are occasional presentations for broader audiences but no all-in-one sort of thing, no
13:57 <legoktm> DannyS712: https://wikitech.wikimedia.org/wiki/User:Quiddity/How_does_it_all_work
13:57 <Majavah> I've got most of my knowledge from just trying to do things in deployment-prep and reading puppet manifests to do things  in deployment-prep
13:57 <DannyS712> and so I guess what I should research is what exactly is puppet
13:58 <apergos> I have a writeup on puppet for dumps folks but some of it is going to be irrelevant to you... 
13:58 <mutante> DannyS712: https://en.wikipedia.org/wiki/Puppet_(software)
13:58 <cscott-away> DannyS712: it's a configuration management system for installing servers
13:58 <DannyS712> because I understand how most of the code in mediawiki works, and can probably undestand how mediawiki interacts with the database, etc., but there is so much more
13:58 <apergos> https://www.mediawiki.org/wiki/SQL/XML_Dumps/Puppet_for_dumps_maintainers
13:59 so the examples are very dumps specific but the info is maybe general enough to be useful. 
13:59 <cscott-away> DannyS712: Restbase is tricky because it was designed for one use and then has evolved over time into other uses and is "in theory" being replaced but i'm not sure i believe it.
13:59 <apergos> it was
14:00 <Majavah> was?
14:00 * apergos shuts up because my mother told me, if i can't say anything nice, not to say anything at all
14:00 <apergos> yes, restbase is being replaced, work is being done towards that end, slowly but surely
14:00 <cscott-away> DannyS712: the original idea of restbase was as a sort of push-based parser cache -- instead of waiting until an article is fetched/pulled and then parsing it, we'd push updates into restbase from parsoid on every edit, and on template edits push updates to affected articles via a ChangePropagation service.
14:01 <cscott-away> DannyS712: this helped with visualeditor startup, because the "editable version" was always available; the latency was pushed into "between two successive edits" which is more rare.
14:02 <cscott-away> DannyS712: also once-upon-a-time the place was to use this to provide "true" time-travel/archivability of wikipedia.  We'd store all the edited pages in all the revisions natively in HTML, and built on this new cassandra thing, and gradually replace the main DB for article storage.
14:03 <cscott-away> DannyS712: and then over time it turned out that having a light weight storage system in not-PHP that didn't require going through the main MySQL database was very helpful, and analytics, research, machine learning, mobile, etc hopped on.
14:04 DannyS712: restbase has/had the benefit of a relatively clean modern REST API and modular structure.
14:05 (ie, you didn't have to package your ORES data inside a special page and use the action API to access it)
14:06 <cscott-away> DannyS712: now parsoid is moving out of restbase and (in theory) the main parser cache will gain the ability to hold edits-in-progress and be preloaded like restbase.  and i'll let apergos tell you what's happening with all the other users of restbase because i have no idea.
14:06 DannyS712: and that concludes the tl;dr of RESTBase. :)
14:07 <_joe_> DannyS712: or said otherwise, we put the cart (storage) in front of the ox (the application), which is a bad antipattern, and we've been working on getting rid of it for 5 years
14:07 also a tldr of restbase
For next time I get confused about the docker files