OpenSearch Dashboards (previously known as Kibana) is the frontend for Logstash, available at https://logstash.wikimedia.org.
This page is the user guide for OpenSearch Dashboards at WMF. For information about its operation, see Logstash. To read more about the software and its history, check OpenSearch on Wikipedia.
- Logs from MediaWiki end up here.
$logger = LoggerFactory::getInstance('Flow'); $logger->info(...)in MediaWiki PHP corresponds in Logstash to
type:mediawiki channel:Flow level:INFO
- For more about how to do that within works MediaWiki, see mw:Structured logging.
- Start from one of the blue Dashboard links near the top, more are available from the Load icon near the top right.
- In "Events over time" click and drag to zoom in to a specific region.
- On the top right, you can change the time range to start with, e.g. last 24 hours, last 7 days. Smaller ranges are faster.
- If you get lost, start again from the homepage at https://logstash.wikimedia.org/
- Learn how to review production errors.
- "How to Logstash & Kibana", Timo Tijhof, October 2020. (video, NDA-restricted) (public slides)
MediaWiki dashboard template
The Home page is itself also dashboard. It has a single text panel with a Markdown list of links. Add your own for easy access!
Copy results out of a search
Unfortunately it's not possible to just copy & paste stuff out of a search window (because formatting will break badly). If you feel like you really need the data, you can copy the POST request via firefox dev tools (as cURL) and then:
# "log" being the field to print from each document curl ... | gunzip | jq -r '.rawResponse | .hits | .hits | .._source | "\(.timestamp) \(.log)" | gsub("[\\n\\t]"; "")'
The browser address bar for Logstash navigations, are by default personalised to your login session. Sharing such link with other people, leads to a "Unable to restore URL" error.
To share results with others, use the "Share" link from the top right navigation, choose "Permalinks", enable "Short links", and then press "Copy link".
By default, shared links use the "Snapshot" mode which means it captures the state of the dashboard queries and panels as-is. This includes e.g. the timestamp slider, so if you're viewing "Last 1 hour" then the shared link will show different results an hour from now.
To share a specific result, use the single document link instead. Expand one of the raw events in the feed down on the dashboard, and copy the "View single document" link.
- If you see no events at all, perhaps you are querying the future only?
- If you see no results or the results seem unrelated, press the magnifying glass at right of the main query bar to submit again. There is a race condition where if you modify the query while it is running, it ends up re-submitting the last completed query instead of the one you just wrote.
- If you see events suddenly stop, perhaps the query includes the future (e.g. "Today" and "This week" instead of "Last 24h" or "Last week").
- If you think you found when results first started to match your query, double check if it aligns with "today - 90 days ago" which is our message retention.
- The visualisation panels are re-usable and thus saved globally.
- Avoid changing existing visualisations unless intending to change other dashboards that use it at the same time. Otherwise "Save as.." under a new name first.