Summary

We're moving from developing features to polishing then on the Search Update Pipeline. And testing the various requirements for our production deployment. Deployment of the Search Update Pipeline and migration of all wikis to the new pipeline will happen next quarter.

We are working through optimizing the performance of our improvement to multilingual zero-results rate.

What we've accomplished

Search Update Pipeline

Ongoing testing of Flink / Zookeeper integration - https://phabricator.wikimedia.org/T342149
Ongoing refactoring to extract common configuration and graph factories
Alignement of WDQS Streaming Updater with current best practices - https://phabricator.wikimedia.org/T326914
Add a way to filter input events per wiki - https://phabricator.wikimedia.org/T345634
Consider dropping support for java8 - https://phabricator.wikimedia.org/T346015

Improve multilingual zero-results rate

First draft of implementing the analysis improvemnts a

Operations

Search incident during the datacenter switch. Performance was decreased enough so that we dropped search queries. The system recovered mostly on its own once caches were warmed up. We're learning things and we will be better prepared for the next one. - https://wikitech.wikimedia.org/wiki/Incidents/2023-09-20_Elasticsearch_unavailable

Misc

Replaced the use of "whitelist" with "allowlist" in the WDQS code base (this follows a similar change to our puppet code - https://phabricator.wikimedia.org/T343856) - https://phabricator.wikimedia.org/T344284