Benchmark Selenium Test Executions on Machines with Different OS and CPU

Choose cost-effective build machines for executing automated tests

6 min readJun 20, 2022

A repost of my daughter’s article with permission and slight modification. I added it here for the convenience of my blog subscribers.

Execution speed is an important factor in Automated Functional UI Testing, especially for a large regression suite (e.g. 200+ user story level tests, Level 3 of AgileWay CT Grading). While we can run the tests in a Continuous Testing server that supports real (not in test automation framework, which is bad) parallel test execution (like BuildWise), it is still wise to choose a fast, reliable, and affordable build machine for executing automated tests.

In this article, I will benchmark test executions of the same Selenium WebDriver test in the following build machines (in my testing lab):

MacMini 2012 — i7
iMac 2015 — i5
MacMini 2020 — M1 (the base model)
Windows 10 — in a VM inside iMac 2015
Linux Mint — in a VM inside iMac 2015

Test Case: a Selenium WebDriver test in RSpec. This is an end-to-end test for WhenWise covering a wide range of web elements in a typical modern web app.

Execution: in TestWise IDE on macOS, Windows, and Linux.