Search Platform/Weekly Updates/2023-09-01

Summary

Not much happening this week.

What we've accomplished

Improve multilingual zero-results rate

• Ongoing investigation in the increased indexing cost related to the new analysis configuration

Search Update Pipeline

• We now have metrics on duplicate files in Search results (~300 duplicates/day vs ~30'000'000 search/day), which is low enough that we can ignore this use case at the moment. https://grafana-rw.wikimedia.org/d/hOONJnkIz/cirrus-search-deduplication - https://phabricator.wikimedia.org/T341227

Misc

• Permissions are now correctly set on new snapshots discovery.wikibase_rdf https://phabricator.wikimedia.org/T342416
• New elasticsearch dev image is published, with the latest version of our custom plugins - https://phabricator.wikimedia.org/T344841
• We worked with Addshore to publish a new Wikidata Query Service journal, which simplifies the process for people who want to run their own query service. See the write up by Addshore: https://addshore.com/2023/08/wikidata-query-service-blazegraph-jnl-file-on-cloudflare-r2-and-internet-archive/ - https://phabricator.wikimedia.org/T344732