Model & Performance: We're using Moto G5 and Samsung A51 Android phones. They reflect the CPU speed of our primary user base in certain countries. In the future we also want to run Apple devices.
Root Access: Some phones need to be rooted. This allows us to control the CPU speed consistently, aiding in stable performance metrics.
Application Updates: Regular updates are crucial and we need to able to choose exact when those happen. We upgrade Chrome and Firefox upon each new stable release.
Screen Monitoring: We utilise scrcpy for screen viewing. That helps us update and find problems on the phone.
Temperature Control: To avoid performance changes due to heating, we monitor and control the battery (and by extension, CPU) temperature. You need to make sure that the phones are cold.
No cellular service: The phones do not need any cellular service. They will use either a wifi or the internet connection from the server.
Wifi connection: Preferable the phones can connect to a wifi. That is good when we need to run upgrades and then can use the Play store.
Servers
One Server Per Phone: Ideally, each phone is tethered to a dedicated server. This setup allows for independent and varied connectivity settings.
Server Specifications:
Can be Raspberry Pi or any Linux-based system (ARM or AMD).
Requires USB connection to phone.
Ethernet access with internet.
We need to be able to access the server using SSH
Data Handling: Servers will upload data to S3, send metrics to Graphite, and use Redis for server-to-server communication.
Raspberry Pi Usage: For Raspberry Pi servers, VNC is used for screen access.
Network
Stability & Realism: We aim for a consistently stable network, simulating 3G/4G conditions by controlling speed and adding delays.
Connection Methods:
Tethering from the server is preferred for stability. We use Linux Traffic Control (tc) on the tethered connection.
We tried sharing the WiFi from Raspberry Pis and that works fine but if there's many WiFis it seems to be good to have one phone per WiFi source.
We tested a Netropy device but tethering or Raspberry Pi WiFi gets us more stable network performance.
Example setup and workflow
Example setup
Previous Setup Example: We ran tests on phones accessing Wikipedia, driven by a server running sitespeed.io. Tests, configurations, and URLs were managed from a public Git repository. Metrics were sent to Graphite, with test results stored on S3. WebPageReplay was also used for stable metric comparisons.
Upcoming Changes: Some servers will connect to Redis for new job assignments.
