Definition of End-to-End Test Automation Success

Measuring test automation attempts objectively. Only after knowing and admitting what went wrong, people can learn and succeed.

Test Automation failures are everywhere.

Here are comments I heard personally while working in three large IT divisions (each with 200+ staff, and numerous agile coaches/scrum masters) in my city.

• “This company had four attempts of test automation. Every single time, in the beginning, people were excited. Soon nobody is interested” — a veteran tester
• “For the last 12 years in this company, test automation attempts all failed. I admit I don’t know about test automation. I welcome anyone’s suggestions.” — a test lead said in a meeting after the company’s new $3 million test automation attempt failed.
• “Maintenance of these Gherkin automated tests cost three times of our development effort (coding).” — a manager (and test automation was dumped later, but the principal software engineer who was responsible stayed)