WebPageTest/UpdateToLinux
Replaced with WebPageReplay. 2022
Background
In June (or July) 2018 our current WebPageTest Windows agent will stop to work because of changes in Chrome and the old agent will not be upgraded to work with the change, meaning we need to move to the new agent multi platform agent.
The new multi platform agent works both on Windows and Linux. We've been trying out the new one on Linux for on and off for half a year and filing upstream bugs and soon it is time for us to do the switch.
We will move our agent to Linux and there are a couple of reasons: The Linux agent is cheaper to run on AWS (than Windows), Linux makes it easier for us to test out ourself and do pull requests if we fix bugs. And it also makes it possible for us to move the agent in-house in the future. And we can add more testing for more Wikis and more pages.
We will miss out on performance tests on Internet Explorer, however our old agent couldn't be updated so we are testing Internet Explorer 11 with no HTTP/2 support.
Status
Testing the new Linux version has been going on for a long time and it tracked in https://phabricator.wikimedia.org/T165626
Timeplan
We aim to remove the Windows agent the 26 of March 2018.
Known issues
There are a couple of bugs and they are tracked in https://phabricator.wikimedia.org/T186466.
What's important for you as a WebPageTest user is that the Linux instance seems faster than before (most of our pages now have faster first visual change and lower Speed Index values).
What also good to know is that for some pages on desktop we have seen a bigger variance than before. However for our alerting we are aiming to move to our WebPageReplay setup.
Migrating your WebPageTest Grafana dashboards
Migrating your old dashboards to start using the metrics from our new Linux agent is pretty straight forward. We have changed the name of the location from us-east-1 to us-east. That means that in all your dashboards you need to change us-east-1 to us-east.
There's one annoying thing and that is that the Chrome browser names has changed between the old WebPageTest agent and the new one. That means that you need to be careful when you do the change in Grafana because key parts will not be populate themselves when you change the location. It's easiest to Toggle edit mode and use the plaint text mode to do the change.
The browser names changes:
Google_Chrome -> Chrome
Google_Chrome-emulateMobile -> Chrome-emulateMobile
Firefox are the same.
You can checkout https://grafana.wikimedia.org/d/000000210/webpagetest as an working example of a dashboard that moved to the new Linux agent.