My Innovative Solution to Continuous Testing: Parallel Automated End-to-End Test Execution

A case study of how I implemented parallel test execution for Automated End-to-End tests (via UI) in 2008.

This is included in the “My Innovative Solution to Test Automation and Continuous Testing” series.

Table of Contents:
· Challenge: Automation Regression Testing’s Growing Time
· The idea of Parallel Test Execution
· Customize CruiseControl
· Parallel Testing Tab
· Stats
· Birth of BuildWise

On 2022–08–31, Apple announced Xcode Cloud, a continuous integration and delivery service. One of its highlighted features is “run automated tests in parallel”.

Image Credit: https://developer.apple.com/xcode-cloud/

I came up with the idea of parallel execution of automated end-to-end tests and implemented it in a CI server back in 2008. At that time, to my knowledge, there was no other doing it.

14 years ago, there were only two CI servers existed: CruiseControl (by ThoughtWorks) and Hudson (predecessor of Jenkins). CruiseControl was more well-known in Australia back then. In terms of test execution, these two CI servers were designed for executing only Unit and Integration tests (in JUnit).

Challenge: Automation Regression Testing’s Growing Execution Time

I set up a CruiseControl server for the project to run our Watir test suite. It worked quite well for a month or two. The team could trigger a build easily, view changelog, and build results, all on CruiseControl’s web interface. Then, we could act quickly based on testing feedback.

The team members initially were in doubt, but later on, all embraced automated regression testing for its obvious benefits, the most valuable one: many regression errors were detected early. (these java programmers started to learn and love Ruby). Therefore, the number of test cases grew quickly. With more tests being created, there came longer test execution time. At first, I didn’t pay much attention to this situation. Anyway, it was my first test…