Irrational and False Excuses for Web Test Automation Failures

Real reason: Incompetence. Just admit ‘lack of the skills’, change the mindset, and seek professional help if necessary.

  • “Tests failed because the application changed”
  • “The test scripts are fine, only test data is no longer valid”
  • “Our testers do not have the capability”
  • “The automation framework selected by the previous test architect was bad”
  • “Automated end-2-end tests are known as fragile”
  • Long execution time (comparing to unit/API tests)
  • Testing for all web applications, technically speaking, is the same. If one tester can do test automation successfully at one job, he shall be able to replicate the success on other jobs, with virtually no changes.
  • free (in both freedom and cost)
  • the only automation framework supported by all major browser vendors
  • popular, open-source and well maintained
  • used and recommended by top IT companies such as Facebook, Microsoft and Google.

1. “It worked well on my last project

This is a “Dog ate my homework” level lie used by naughty children. My question is “How about this project?” Show me something today, then tomorrow, and onwards. A real test automation engineer can implement a few key tests on day 1. The reason is simple: web test automation is pretty much the same for all projects.

2. “Test execution failed because the application changed”

What?! Is “application changes” a surprise to you? Come on, this is normal. How could a software/test engineer even say that? If the application never changes, there is no need for regression testing.

3. “The test scripts are fine, only test data is no longer valid”

Wrong, test data is a part of test scripts.

4. “Our testers do not have the capability”

This is often used by managers and tech-leads (such as Test Architect, Chief/Principal software engineers). Yes, it was the fact. However, have you done anything to improve the situation?
Moreover, do the managers/tech-leads know what are the skills/capabilities required? I doubt it.

5. “Test Automation is expensive, a XXX tool/runtime license costs $$$$”

Why don’t use the 100% free and future-proof Selenium WebDriver? After all, it is the only one that is W3C compliant. Facebook and Microsoft recommended it.

Image credit: from the slides in the video presentation

6. “Still working on some issues of our own-framework, …”

Crazy. First of all, very very few engineers in this world can create a new working framework. This so-called “own-framework” is normally just an unnecessary layer on top of Selenium. From my experience, each of these ‘new frameworks’ failed for a silly and simple reason: this badly-designed unnecessary layer was buggy.

7. “Selenium WebDriver has a steep learning curve”

Totally wrong! Selenium WebDriver is the easiest to learn test automation framework. Most people I mentored could work on real tests after one-day training. My 12-year-old daughter learned to write the raw Selenium WebDriver tests, so could you and your teenage child under proper mentoring.

8. “The reason we have to use Java/JS/C# is that we have the skills to maintain”.

Untrue. Most people will agree that, compared to on-going maintenance, creating automated tests is a lot easier. Now check the reality. After several months, the team might have been struggling with test creation by using the ‘preferred’ language. There is no point to talk about reaching the finishing line if already failed at the starting line.

9. “Selenium WebDriver execution is not stable”, “Selenium could not test AJAX”, …

Totally untrue. Selenium WebDriver is the most stable web automation framework. Remember, web test automation depends on browsers, and all major browser vendors only support the WebDriver standard. You may use ‘unstable’ to describe any other framework such as UTP and Cypress, but not Selenium WebDriver.

10. “The automation framework selected by the previous test architect was bad”

A typical office politics. Though it might be true, a test architect is only allowed to use this excuse for one day. Web test automation skills and practices are fully transferable. A real test automation engineer, like me and my teenage daughter, can implement a few key tests on the first day.

11. “Automated end-2-end tests are known as fragile”

Yes, that’s why you are hired to implement it. It is still professional for engineers to reject test automation because they are unable to accomplish the task. However, if someone did take the task (and getting paid) and failed it, using this excuse is quite low.

12. “Our testers have difficulty using Java/JavaScript, the language used in test scripts.”

Why not change to Ruby? Ruby is the best scripting language recommended by the classic ‘Agile Testing’ book. The real reason: Test Lead does not know Ruby, and lacks a learning altitude, which is more concerning.

13. “The automated UI test scripts are hard to read, difficult to maintain by nature”

Not necessarily. Writing tests in RSpec (Ruby) will greatly enhance the readability of the test scripts. I could maintain 617 and 500 selenium tests for my two apps ClinicWise and WhenWise, respectively. Of course, efforts are required but maintenance is surely possible. Otherwise, why is your company trying to do it anyway?

14. “The DevOps team is no good, unable to run our tests reliably in Jenkins”

I will write another article on silly and false excuses for CI/CD/CT failures. Here I would like to focus on test automation. My question is: “How reliable are your tests?” If your team members run the same test script at different times, are you confident to get 10 successful test executions?

UI test execution Stats of WhenWise on BuildWise CT server

Award-winning software developer, test automation & CT coach, author, and speaker. Help teams succeed with Agile/DevOps by implementing real Continuous Testing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store