mwlog

mwlog are a class of logging hosts which serve multiple functions.

Hosts

• mwlog1002 (Eqiad)
• mwlog2002 (Codfw)

Services

udp2log/mwlog

Ingest and storage of "udp2log" logs, in flat file format in /srv/mw-log.

These are logs transported over udp/8420 to an rsyslog listener on the mwlog hosts which mirrors the logs to the udp2log listeners on all mwlog hosts (udp/8421)

udp2log/mwlog producers are configured in various different ways:

• Mediawiki uses hard coded IPv4 addresses (intentionally, for performance reasons) for the mwlog destinations, configured in mediawiki-config/wmf-config/ProductionServices.php
• Some services (including rsyslog which relays on behalf of php-fpm and apache2 ) use the udplog.(eqiad|codfw).wmnet DNS records. Updating the udplog DNS record will require an rsyslog restart across the fleet.
• Some services are defined via puppet hiera (can be found with e.g. git grep mwlog1002 in operations/puppet)

Long-term security storage

Currently /srv/security on the mwlog primary (eqiad) is used as a long-term store for security log archives.

Former services

• Redis instance for Arc Lamp (2014-2023).

Utilities

• logspam-watch displays and filters a summary of recent logs from disk - see Wikimedia binaries#mwlog-server