My Continuous Testing Journey

Execute automated test suite from Testing IDE ⇒ Build Scripts from Command Line ⇒ CI Server ⇒ CT Server

Image Credit: http://getdrawings.com/journey-drawing#journey-drawing-35.jpg

Non-Medium-Members: you can view this article free on Vocal Media.

My approaches to running all automated UI tests (as regression testing) have gone through the following stages:

• 2006–2007: From IDE. (here I mean running all regression tests. For developing/debugging individual automated tests, I have always been using TestWise since 2006)
• ~2007: From the command line (build script)
• 2007-2008: In a CI Server: CruiseControl
• 2009–2012: Customized CruiseControl with parallel execution, dynamic ordering, …
• 2012 — present: In my own Continuous Testing server BuildWise (BuildWise was made available to the public in 2017)

1. Run tests from IDE (2006–2007)

As a programmer (at that time), I naturally tried to run functional UI tests in IDEs (including NetBeans and my own TestWise). However, it didn’t take long for me to realize that it was not right. Functional (UI) tests, compared to unit tests, take a much longer time to execute. For example, a small suite of 20 tests of which each test has an average of…