Continuous Testing Clarified

Continuous Testing is the most important process in DevOps; Yet most so-called ‘Agile’ or ‘DevOps’ projects did not have it or got it wrong.

Non-Medium-Members: you can view this article free on Vocal Media.

This article is one of the “IT Terminology Clarified” series.

According to Wikipedia, continuous testing is “the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.” The keywords are “automated tests”, “delivery pipeline”, “immediate feedback” and “software release candidate”. (The reason I quote Wikipedia, a non-academic reference, for definitions: because Continuous Testing is new, and there is a lot of confusion over it. So I reach for a common understanding at Wikipedia)

Table of Contents:
· Continuous Testing explained in simple words
· Continuous Testing is the Holy Grail of Software Development
· Software Development Trend ⇒ Continuous Testing
· Continuous Testing vs DevOps
· Continuous Testing vs Continuous Integration
· If CI is implemented properly, no need for CD or CT
· Continuous Delivery is really about testing
· Reality Check of CT/DevOps

Continuous Testing explained in simple words

Like many other formal technology definitions, the above (on Wikipedia) sounds right but is not quite clear. Let me interpret this:

• “executing automated tests”, “business risks”
  The automated tests are at the user story level: testing business features. For example, for a web app, the CT process will run a set of automated test scripts to drive the app to verify business functions, in a browser.
  
  Comparatively, unit testing is conducted by programmers only, that’s why unit tests are also called programmer tests.
• “pipeline”, “immediate feedback”
  In this pipeline, Customers/Business Analysts and programmers are waiting for feedback, and more importantly, ready to act on the feedback. Modification follows feedback, i.e, if there are test failures, a new build (with potential fixes) will be triggered to run another execution of…