Regression Testing Clarified

Regression testing needs to be automated and run in a Continuous Testing server multiple times a day.

Non-Medium-Member: Read this article free on Vocal.

This article is one of the “IT Terminology Clarified” series.

Between 2010–2013, I spoke at a few international software testing conferences (and decided no more, see reasons here). Quite commonly, there were pre-attendance surveys on the attendants’ interested topics. From my memory, the top three (the order might vary) were the same.

• Test Automation (End-to-End)
• Regression Testing
• Web App Testing

However, topics on regression testing were rare at software testing conferences. Several attendants told me this at different events (after attending mine).

In this article, I will share my thoughts and experience in regression testing. This article is a bit long, with the following sections:

• What is Regression Testing?
• Status of Regression Testing in Software Projects (quite sad)
• Regression Testing must be automated, however, …
• Manual Regression Testing is Evil in Agile
• How often do you do regression testing? (correct answer: multiple times a day)
• How long should full regression testing take? (correct answer: about 30 minutes)
• Why don’t software teams do automated regression testing?

For readers who are interested in the implementation, check out this article: How to Implement Real Automated Regression Testing?

1. What is Regression Testing?

Regression Testing in software means “re-running functional and non-functional tests to ensure that previously developed and tested software still performs after a change” (Wikipedia). Every software team member understands the objective of regression testing, as we often see regression errors. However, not many software engineers (including testers) understand how to do regression testing properly.