Analytics/Systems

These subpages explain in technical detail the systems that process data for Analytics at Wikimedia Foundation. They include information about setup, maintenance, architecture, and more.

Child Pages of Analytics/Systems

AQS · Airflow · Anaconda · Archiva · Clients · Cluster · DB Replica · Dashiki · DataHub · Dealing with data loss alarms · Druid · EventLogging · EventStreams · Event Data retention · Event Sanitization · Exporting from HDFS to Swift · Geoeditors · Hive to Druid Ingestion Pipeline · Jupyter · Jupyter-SWAP · Kerberos · Maintenance Schedule · Managing systemd timers · Manual maintenance · MariaDB · Matomo · Presto · Refine · Reportupdater · Siege · Superset · Tier2 · Turnilo · Varnishkafka · Wikimetrics · Wikistats · Wikistats 2 · ua-parser

---

All Subpages of Analytics/Systems

• AQS
• AQS/Scaling
• AQS/Scaling/2016/Hardware Refresh
• AQS/Scaling/2017/Cluster Expansion
• AQS/Scaling/2020/Cluster Expansion
• AQS/Scaling/LoadTesting
• Airflow
• Airflow/Airflow testing instance tutorial
• Airflow/Developer guide
• Airflow/Instances
• Anaconda
• Archiva
• Clients
• Cluster
• Cluster/AMD GPU
• Cluster/Beeline
• Cluster/BrowserReports
• Cluster/Coordinator
• Cluster/Data Format Experiments
• Cluster/Data deletion and sanitization
• Cluster/Deploy/Refinery
• Cluster/Deploy/Refinery-source
• Cluster/Edit data loading
• Cluster/Edit history administration
• Cluster/Edit serving layer
• Cluster/Geolocation
• Cluster/Geotagging
• Cluster/Gobblin
• Cluster/Hadoop
• Cluster/Hadoop/Administration
• Cluster/Hadoop/Alerts
• Cluster/Hadoop/Load
• Cluster/Hadoop/Test
• Cluster/Hive
• Cluster/Hive/Alerts
• Cluster/Hive/Avro
• Cluster/Hive/Compression
• Cluster/Hive/Counting uniques
• Cluster/Hive/Queries
• Cluster/Hive/Queries/Wikidata
• Cluster/Hive/QueryUsingUDF
• Cluster/Kafka/Capacity
• Cluster/Mediawiki History Snapshot Check
• Cluster/Mediawiki history reduced algorithm
• Cluster/Mysql Meta
• Cluster/Oozie
• Cluster/Oozie/Administration
• Cluster/Page and user history reconstruction
• Cluster/Page and user history reconstruction algorithm
• Cluster/Revision augmentation and denormalization
• Cluster/Spark
• Cluster/Spark/Administration
• Cluster/System Users
• Cluster/Workflow management tools study
• DB Replica
• Dashiki
• Dashiki/Configuration
• DataHub
• DataHub/Upgrading
• Dealing with data loss alarms
• Druid
• Druid/Alerts
• Druid/Load test
• EventLogging
• EventLogging/Administration
• EventLogging/Architecture
• EventLogging/Backfilling
• EventLogging/Data representations
• EventLogging/EventCapsule
• EventLogging/Monitoring
• EventLogging/NotErrorLogging
• EventLogging/Outages
• EventLogging/Performance
• EventLogging/Publishing
• EventLogging/Sanitization vs Aggregation
• EventLogging/Schema Guidelines
• EventLogging/Sensitive Fields
• EventLogging/TestingOnBetaCluster
• EventLogging/User agent sanitization
• EventStreams
• Event Data retention
• Event Data retention/AppInstallId
• Event Sanitization
• Exporting from HDFS to Swift
• Geoeditors
• Hive to Druid Ingestion Pipeline
• Jupyter
• Jupyter-SWAP
• Jupyter/Administration
• Jupyter/Tips
• Kerberos
• Kerberos/UserGuide
• Maintenance Schedule
• Managing systemd timers
• Manual maintenance
• Manual maintenance/Refined flags script
• MariaDB
• Matomo
• Presto
• Presto/Administration
• Presto/Query Logger
• Refine
• Reportupdater
• Siege
• Superset
• Superset/Date functions
• Tier2
• Turnilo
• Varnishkafka
• Wikimetrics
• Wikimetrics/Adding New Features
• Wikimetrics/Adding New Features/CentralAuth Cohorts
• Wikimetrics/Adding New Features/Tag Cohorts
• Wikimetrics/Global metrics
• Wikistats
• Wikistats/Traffic
• Wikistats2/Metrics/FAQ
• Wikistats 2
• ua-parser
• ua-parser/2019-09-18 Update