Continuous Testing Clarified

Continuous Testing is the most important process in DevOps; Yet most so-called ‘Agile’ or ‘DevOps’ projects did not have it or got it wrong.

Zhimin Zhan
10 min readNov 16, 2021

Non-Medium-Members: you can view this article free on Vocal Media.

This article is one of the “IT Terminology Clarified” series.

According to Wikipedia, continuous testing is “the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.” The keywords are “automated tests”, “delivery pipeline”, “immediate feedback” and “software release candidate”. (The reason I quote Wikipedia, a non-academic reference, for definitions: because Continuous Testing is new, and there is a lot of confusion over it. So I reach for a common understanding at Wikipedia)

Table of Contents:· Continuous Testing explained in simple words
Continuous Testing is the Holy Grail of Software Development
Software Development Trend ⇒ Continuous Testing
Continuous Testing vs DevOps
Continuous Testing vs Continuous Integration
If CI is implemented properly, no need for CD or CT
Continuous Delivery is really about testing
Reality Check of CT/DevOps

Continuous Testing explained in simple words

Like many other formal technology definitions, the above (on Wikipedia) sounds right but is not quite clear. Let me interpret this:

  • executing automated tests”, “business risks
    The automated tests are at the user story level: testing business features. For example, for a web app, the CT process will run a set of automated test scripts to drive the app to verify business functions, in a browser.

    Comparatively, unit testing is conducted by programmers only, that’s why unit tests are also called programmer tests.
  • pipeline”, “immediate feedback
    In this pipeline, Customers/Business Analysts and programmers are waiting for feedback, and more importantly, ready to act on the feedback. Modification follows feedback, i.e, if there are test failures, a new build (with potential fixes) will be triggered to run another execution of…

Zhimin Zhan

Test automation & CT coach, author, speaker and award-winning software developer. Help teams succeed with Agile/DevOps by implementing real Continuous Testing.