How to Rescue a Failed Test Automation?
Tips on how to rescue failed test automation technically, and more importantly, address the subtle human factors
--
Update (2022–10–04): Case Study: Rescue Unreliable 20 hours of Automated Regression Testing in Jenkins ⇒ 6-Minute Highly-Reliable in BuildWise CT Server
Over the last 10 years, I have successfully rescued many failed test automation in different software projects using the same technical formula. I was unable to rescue some as well, due to human factors. In this article, I will share my thoughts and experience.
First of all, compared to the green-field project, it is not going to be easy with an existing failed approach in place. There are a number of human factors.
Table of Contents: ∘ Technically Competent
∘ Human factors on existing failed automation solution
∘ Talk Test Automation in the context of "Release Often" to upper management
∘ Talk Test Automation in the context of "Automated Regression Testing" to team members
∘ People tend to be fixated on a tool; often, they want to be that way
∘ Replacing with another "Tool"
∘ Specific Advice
Technically Competent
Testing is practical, automated testing means practical and objective (clearly measurable with no grey area). To rescue a failed test automation (being a hero), you need to be technically competent first. I mean real competence here. Try to answer these questions:
For your last project,
- What's the size of the E2E test suite?
- How often do you run them?
- Do they all pass?
- How long does it take?
- How often does your team push updates to production?
I can answer the above (except the last one) with one screenshot.
My answer to the last question: once I get a green build (passing all UI tests, like the one above), I push it to production immediately. Quite often, multiple…
