WMDE/Analytics/WMDE Anayltics Graphite Metrics

From Wikitech

This page is

  • where WMDE dashboard owners state the description of new dashboards or indicators that need to be added to Grafana;
  • where WMDE Software Development documents all MediaWiki objects (metrics) that get passed to Graphite from their code, and
  • where WMDE Analytics learn about the respective needs and metrics in order to query Graphite correctly and represent the results on Grafana appropriately.

Best Practices for Documenting Graphite Metrics

Best practices [Section to be expaned]

Dashboard Feature Template

Name of the dashboard component (e.g. TwoColConflict page views from the TwoColConflict dashboard)

  • description: this is where the dashboard owners explain what indicator(s) should be reported upon (visually and numerically)
  • metrics: this is where the developers explain what are the names of the metrics in Graphite that are needed to meet the demand as given in the description. Occasionally, the developers also needs to provide the semantics, i.e. to explain the meaning of the metric, for example if we are beginning to use a previously unused metric.
  • implementation: this is where the analytics reports back on the implementation of the new component of the dashboard.
  • query: this is the complete Graphite query as used in Grafana to collect the desired statistics and turn them into the indicator(s) that matches the needs as provided in the description.

WMDE Grafana Dashboards

This is where we list all Grafana dashboards that need maintenance.

The suggested workflow:

  • the owner of the dashboard (typically Technical Wishlist, New Editors, Wikidata teams) define the description, stating as precisely as possible what the new indicator on Grafana should be telling;
  • the developer of the new feature that need Graphite tracking provide relevant metrics and their semantics (i.e. what do they mean in reference to the description, so that the analytics can for example know how to combine several metrics in order to obtain the result that matches what is desired in the description; NOTE: most of the time, the semantics of the metrics are intuitively clear provided additional information in the description, so there's no need to go into detail here, although it would be important explain the semantics of the newly introduced metrics);
  • while the analytics provide the respective Graphite query and describe the implementation on the respective Grafana dashboards.

Mediawiki TwoColConflict

Dashboard Description

The dashboard tracks the usage of the TwoColConflict MediaWiki extension features.

Grafana Components

  • TwoColConflict page views
    • description: How many people saw the TwoColConflict page (in total).
    • metrics: MediaWiki.TwoColConflict.conflict.sum
    • implementation: Singlestat
    • query: summarize(MediaWiki.TwoColConflict.conflict.sum, '1d', 'sum', false) - provide semantics (what does this mean)
  • Default page views
    • description: How many people saw the Default page (in total).
    • metrics: MediaWiki.edit.failures.conflict.sum - provide semantics (what does this mean), MediaWiki.TwoColConflict.conflict.sum
    • implementation: Singlestat
    • query:
      • (A) summarize(MediaWiki.edit.failures.conflict.sum, '1d', 'sum', false),
      • (B) summarize(MediaWiki.TwoColConflict.conflict.sum, '1d', 'sum', false),
      • (C) diffSeries(#A, #B)
  • Total Edit Resolution Page Views
    • description: This will include the conflicts that result in the TwoColConflict page being shown.
      • for example, the above is not a good description; the description should state something like "This is the sum of all edit resolution page views, including the conflicts that result in the TwoColConflict page being shown. "
    • metrics: MediaWiki.edit.failures.conflict.sum
    • implementation: Singlestat
    • query: summarize(MediaWiki.edit.failures.conflict.sum, '1d', 'sum', false)
  • Percent Resolved w. TwoColConflict
    • description: Percentage of people who clicked on "publish" on the two col conflict page, averaged out for the time period of 1 day.
    • metrics: MediaWiki.TwoColConflict.conflict.sum, MediaWiki.TwoColConflict.conflict.resolved.sum- provide semantics (what does this mean)
    • implementation: Singlestat
    • query:
      • (A) summarize(MediaWiki.TwoColConflict.conflict.sum, '1d', 'sum', false)
      • (B) summarize(MediaWiki.TwoColConflict.conflict.resolved.sum, '1d', 'sum', false)
      • (C) divideSeries(#B, #A)
  • Percent Resolved w. Default View
    • description: Percent of conflicts resolved in the Default view, averaged out for the time period of 1 day.
    • metrics: MediaWiki.edit.failures.conflict.resolved.sum, MediaWiki.TwoColConflict.conflict.resolved.sum, MediaWiki.edit.failures.conflict.sum, MediaWiki.TwoColConflict.conflict.sum
    • implemenation: Singlestat
    • query:
      • (A) diffSeries(summarize(MediaWiki.edit.failures.conflict.resolved.sum, '1d', 'sum', false), summarize(MediaWiki.TwoColConflict.conflict.resolved.sum, '1d', 'sum', false))
      • (B) diffSeries(summarize(MediaWiki.edit.failures.conflict.sum, '1d', 'sum', false), summarize(MediaWiki.TwoColConflict.conflict.sum, '1d', 'sum', false))
      • (C) divideSeries(#A, #B)
  • Percent Conflicts Resolved
    • description: Percent of total conflicts resolved, averaged out for the time period of 1 day.
    • metrics: MediaWiki.edit.failures.conflict.resolved.sum, MediaWiki.TwoColConflict.conflict.resolved.sum, MediaWiki.edit.failures.conflict.sum, MediaWiki.TwoColConflict.conflict.sum
    • implemenation: Singlestat
    • query:
      • (A) summarize(MediaWiki.edit.failures.conflict.sum, '1d', 'sum', false)
      • (B) summarize(MediaWiki.edit.failures.conflict.resolved.sum, '1d', 'sum', false)
      • (C) divideSeries(#B, #A)
  • BF Disables
    • description: Number of TwoColConflict BetaFeature users that have disabled the feature in the last day.
    • metrics: daily.betafeatures.global_disables.totals.twocolconflict - provide semantics (what does this mean)
    • implemenation: Singlestat
    • query: keepLastValue(daily.betafeatures.global_disables.totals.twocolconflict, 100)
  • BF Users
    • description: Global users
    • metrics: daily.betafeatures.global_user_counts.totals.twocolconflict - provide semantics (what does this mean)
    • implemenation: Singlestat
    • query: keepLastValue(daily.betafeatures.global_user_counts.totals.twocolconflict, 100)
  • TwoColConflict conflicts
    • description: NONE
    • metrics: MediaWiki.TwoColConflict.conflict.sum
    • implemenation: Graph
    • query: alias(MediaWiki.TwoColConflict.conflict.sum, 'conflicts')
  • Beta Feature Users
    • description: ###### Users   - user_counts - The number of individual users that have enabled the beta feature. (If 1 user enabled the feature on 2 wikis they will be counted twice)   - global_user_counts - The number of global users that have enabled the beta feature (If 1 user enabled the feature on 2 wikis they will be counted once)   - global_disables - The number of global users that were using the feature yesterday, but have disabled it today.   - global_enables - The number of global users that were not using the feature yesterday, but have enabled it today.
    • metrics:
      • daily.betafeatures.user_counts.totals.twocolconflict - provide semantics (what does this mean)
      • daily.betafeatures.global_user_counts.totals.twocolconflict - provide semantics (what does this mean)
      • daily.betafeatures.global_enables.totals.twocolconflict - provide semantics (what does this mean)
      • daily.betafeatures.global_disables.totals.twocolconflict - provide semantics (what does this mean)
    • implemenation: Graph
    • query:
      • (A) aliasByNode(daily.betafeatures.user_counts.totals.twocolconflict, 2)
      • (B) aliasByNode(daily.betafeatures.global_user_counts.totals.twocolconflict, 2)
      • (C) aliasByNode(daily.betafeatures.global_enables.totals.twocolconflict, 2)
      • (D) aliasByNode(daily.betafeatures.global_disables.totals.twocolconflict, 2)
  • Resolved %'s over time
    • description: NONE
    • metrics:
    • implemenation: Graph
    • query:
      • (A) summarize(MediaWiki.TwoColConflict.conflict.sum, '1d', 'sum', false)
      • (B) summarize(MediaWiki.TwoColConflict.conflict.resolved.sum, '1d', 'sum', false)
      • (C) summarize(MediaWiki.edit.failures.conflict.sum, '1d', 'sum', false)
      • (D) summarize(MediaWiki.edit.failures.conflict.resolved.sum, '1d', 'sum', false)
      • (E) alias(divideSeries(#B, #A), 'TwoColConflict Resolve %')
      • (F) alias(divideSeries(#C, #D), 'Total Resolve %')
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=WMDE/Analytics/WMDE_Anayltics_Graphite_Metrics&oldid=1832800"