Pixel 2: Visual Regression Testing for MediaWiki

What is Pixel?

Pixel is a visual regression testing tool for MediaWiki that helps developers and QA engineers catch UI regressions before users see them. It automates the process of taking screenshots across different scenarios and comparing them to detect visual changes. Pixel also supports accessibility regression testing using Pa11y and axe-core.

Pixel 2 Improvements

Pixel 2 is a clean-slate rewrite that addresses key architectural limitations of the original

Below you can see an animation of the Pixel 2 results page. It shows results per project. Clicking on a project's heading shows a visual diff viewer where you can compare any differences between reference (latest release branch) and test (main/master) run screenshots for the project's scenarios:

Key Improvements:

Per-project MediaWiki configuration: Each project now has its own isolated configuration, preventing cross-project interference
Centralized configuration: All project settings (extensions, skins, MySQL data, LocalSettings) live in one place per project
Simplified architecture: Eliminated the complex host node layer, making debugging much easier with declarative shell scripts
Provider flexibility: Decoupled from MediaWiki provisioning specifics, allowing different providers (MediaWiki-Quickstart, Codex, etc.)

Beta Status & Feedback

Feedback and suggestions welcome!

Join us on Slack:

#talk-to-test-platform - General questions and feedback
#pixel-development - Development discussions and technical questions

Resources

Live Results

Pixel 1 (previous): https://pixel-old.wmcloud.org
Pixel 2 (current): https://pixel.wmcloud.org

Code Repositories

Pixel 1: https://github.com/wikimedia/pixel
Pixel 2: https://gitlab.wikimedia.org/repos/test-platform/pixel

Running Locally (Quick Start)

Running Pixel 2 locally is straightforward:

Prerequisites: Install Docker

Clone the repo:

git clone https://gitlab.wikimedia.org/repos/test-platform/pixel.git
cd pixel

Run tests:

# Run reference (latest release branch) screenshots
./run "project-name" "reference"

# Run test (latest main/master) screenshots
./run "project-name" "test"

# Or run both and see comparison
./run_reference_and_test "project-name"

For detailed setup instructions, configuration options, and advanced usage, see the full documentation.

Migration Note

If you have an existing project in Pixel 1, it has been migrated to Pixel 2 with the shared configuration copied over. You should review and clean up your project's configuration to include only what's actually needed - doing so will no longer break other projects. See the migration section in the documentation for details.