Search Platform/Documentation

Docs and Other Links

Main Documentation

Search

• Why Search is Important—Why does the Search Platform team even exist?
• https://wikitech.wikimedia.org/wiki/Search/Technical_interactions: how to build features on top of Search
• mw:Help:CirrusSearch—Information on how the Wikimedia search works.
  • Completion Suggester—incremental search
• Search Glossary—a place for definitions, context, and links for terms we use that other people may not be familiar with
• Testing Search—testing search changes is complicated!
• Elasticsearch stats on Grafana
• Logstash for Elasticsearch/Cirrus (requires login)
• Data access and analysis guidelines used by the Search Platform team around data sources, or by other teams around Search Platform data sources, are documented on Meta
• BrowserBot—a browser test bot for search
• Top Unsuccessful Search Queries—The difficulties in creating a list of unsuccessful search results
• mw:Cross-wiki Search Result Improvements
• mw:TextCat—a software component used for doing language detection

Wikidata Query Service (WDQS)

• Project page
• User Documentation
• Runbook
• Streaming Updater (including Runbook)
• Primary WDQS dashboard
• WDQS cluster overview
• Flink Cluster (rdf-streaming-updater) Dashboard

Misc

• Short presentation of the systems managed by the Search Platorm team (11' video, slides)

Blog posts

• Language, Harmony, and Unpacking—A Year in the Life of a Search Nerd (April 2023)
• Permuting Khmer: Restructuring Khmer syllables for search (June 2020)
• Computational knowledge: Wikidata, Wikidata Query Service, and women who are mayors! (March 2020)
• The Anatomy of Search: a series on how full-text search engines work (2018–2019)
  • A token of my affection (August 2018)
  • Variation under nature (September 2018)
  • The root of the problem (November 2018)
  • A place for my stuff (March 2019)
  • In search of… (September 2019)
• Hello, my name is ________: Searching for names is not always straightforward (May 2018)
• Confound it!—Supporting languages with multiple writing systems (March 2018)
• Bare-Bones Basics of Full-Text Search [video] (January 2018)
• Gnomes and trolls and hobgoblins (oh my!)—Failed queries and the vicarious fear of missing out (December 2017)
• So -happy to meet you: Advanced searching techniques on Wikimedia sites (November 2017)
• How we collaborated to build a new open source plugin to improve search results across language-wikis (October 2017)
• Admittedly loopy but not entirely absurd—Understanding our Search Relevance Survey (September 2017)
• Wikipedia, search, and the “цкщтп” keyboard (August 2017)
• Stripping question marks from Wikimedia searches (August 2016)
• Wikipedia seeks to speak your language (July 2016)
• Wikipedia Search Isn’t Necessarily Third BESt (September 2015)

SLOs

• WDQS

Dashboards

Superset

• Search user engagement
• Search user engagement - geolocations
• Search user engagement - Emerging Languages
• Search Metrics
• Fulltext Head Queries

Historical documentation

The Search Platform team was formerly part of the Discovery Department in Audiences; but, as part of the re-organization (tune-up) of June 2017, the Search Platform team is now part of Technology. Pages of historical note:

• Discovery Department (April 2015–December 2017)
• Search (prior to April 2015)