Talk:SLO/Image Suggestions

Review of the current draft:

• It should be clear how does the information from Wikidata get into the Datalake? Until recently, this included the Wikidata dumps and thus also WMDE teams, because they own code in Wikibase that is crucial for creating those dumps. How is this now working? Is there still a dependency on WMDE here that should be mentioned?
• We should less be linking to external websites about a technology in general, but more specifically to the relevant infrastructure at our setup here. For example: Which of the three production Cassandra clusters are we referencing here?
• The teams still need more refinement. For example, the Data Gateway and Cassandra are owned by the SRE-Data Persistence team, not Data Engineering.
• Growth has three API endpoints related to image suggestions:
  • ApiQueryImageSuggestionData which allows adding image-suggestions data to responses from the query Action API endpoint
  • ApiInvalidateImageRecommendation Action API endpoint to allow removing broken suggestions from the pool
  • AddImageFeedbackHandler a REST route to record the user's decision on the recommendations for a given page.
  • To which of those does the 95% SLO apply? To all of them collectively or to each individually or just to one of them?
• Growth monitoring:
  • The "dangling records" that we are monitoring there are exclusively about link recommendations. We do not have any visiblity whether there is a similar issue with image recommendations stored in cassandra and weighted tags in the search index getting out of sync
  • The Growth current monitoring in place is alerting if for 5 minutes the requests for Article Level Image Suggestions (ALIS) is too low: https://grafana.wikimedia.org/alerting/grafana/dencr2n4bt9fkc/view?tab=details
  • The bullet point "GrowthExperiments Action API Performance" is not about the GrowthExperiments Action API and not about image recommendations, but about the independent kubernetes-deployed linkrecommendation service. See its SLO: SLO/linkrecommendation
• Airflow monitoring: Could we link more specifically the relevant panels here? It seems the following are likely relevant, but maybe more?
  • https://airflow-platform-eng.wikimedia.org/dags/ALIS/grid
  • https://airflow-platform-eng.wikimedia.org/dags/SLIS/grid
  • https://airflow-platform-eng.wikimedia.org/dags/cassandra/grid

Michael Große (WMDE) (talk) 07:24, 16 July 2025 (UTC)

@MFossati (WMF) Michael has raised some questions that you're best placed to address with regards to Wikidata. DMburugu (WMF) (talk) 13:10, 21 July 2025 (UTC)

@MGrosse-WMF, DMburugu (WMF):

• first question: we consume Data_Platform/Data_Lake/Content/Mediawiki_content_current_v1, which holds page revisions of all wikis, including Wikidata. We’re now parsing Wikidata page revisions on our own and generate datasets similar to the legacy ones. For instance: before the migration, we were consuming the wmf.wikidata_item_page_link, i.e., Wikidata sitelinks. Now, we extract them by parsing the new dumps and write them to analytics_platform_eng.image_suggestions_wikidata_item_page_link. I'm assuming the dependency on WMDE still holds, but I suggest to double-check that with Data Engineering folks.
• added relevant Cassandra cluster
• teams: I had already updated Data Persistence
• monitoring: added all relevant DAGs

MFossati (WMF) (talk) 17:32, 21 July 2025 (UTC)

@MGrosse-WMF

> ApiQueryImageSuggestionData which allows adding image-suggestions data to responses from the query Action API endpoint and AddImageFeedbackHandler a REST route to record the user's decision on the recommendations for a given page.

These look like the ones which are likely consumed by the web and mobile apps clients so I'd say that the 95% SLO should apply to these. I've added them to the docs.

I've removed the linkrecommendation related information under Growth Monitoring. DMburugu (WMF) (talk) 11:59, 22 July 2025 (UTC)

Thanks both! As I said someplace else, I'd be fine with moving forward with it as it is. We can still iterate on it in the future when we gained some experience with it. Michael Große (WMDE) (talk) 09:04, 23 July 2025 (UTC)