Data Engineering/Evaluations/Event Platform/EventStreams
The Main EventStreams documentation page is at Event_Platform/EventStreams
RCStream Deprecation
RCStream has needed a revamp for a while now. EventStreams is an effort to build a service like RCStream that fits better into our HTTP based architecture, and that can expose more than just the recentchanges stream.
Analytics plans on working with the RCStream community over Q3 to get them to switch to EventStreams, and hope to turn off RCStream by the end of Q4, if not sooner.
Meetings
2016-11-30. RCStream deprecation and Ownership
Attendance: Brandon, Faidon, Mark, Nuria, Otto
Action Items: - Analytics to own communication on RCStream deprecation - Brandon to merge otto's patches regarding service hosting
Rough notes: There are two pieces to this conversation: generic stream service and RCStream[1]. RCStream is understood to be low volume, not really user facing (rather tool-facing). We haver it set up through cache misc . What about the future? Open pipes is an architecture challenge, not cacheable
RCstream data is not cacheable by its very nature. On near term we want to make sure we are supporting just low volume use cases. We see it as a way for people to build tools. Need to document the fact that we only support a low volume use case.
Analytics would own RCStream deprecation and transition to new EventStreams architecture, we will talk to product (who? Roan's team in editing maybe?) to keep them in the loop. We plan on deprecating RCStream in a 6 months timeframe.
Future plans would require an owner from product, let's try to give editing an FYI.
There are issues with RCStream not being https -able (issues with the currently supported socket io libraries), ops was going to do chnages to this extent but we will not be doing those as the migration woudl take care of them.
We also need to clean up the mediawiki end as events are published to the RC streams in different ways. We can probably make IRC recent changes [2] read from kafka and deprecate that piece of publishing code entirely.