E2E Test Automation Anti-Pattern: Using Developers-Coded Utility For Preparing Test Data
A typical software engineer’s thinking, violates the principle of “Black-box/White-box testing”, causes many issues and introduces limitations.
--
On one E2E Test Automation rescue mission, I asked the test engineer to show me how he typically runs an automated E2E test. It took him a minute or so to fiddle with a couple of configuration files before triggering a run. Then, I said, “This was on the dev server, right? Can you make it run against the Test server?”. He said, “OK”, and then went to change some entries in the configuration files. This is Wrong! for E2E Test Automation.
Firstly, we need to understand what “End-to-End Testing” really means. According to the World Quality Report 2018–19, “The first time ever that `end-user satisfaction” is the top objective of quality assurance and software testing strategy”.
The audience of E2E Automation is End-User. While an end-user might not run E2E automation scripts, the scripts must be written from an end-user’s perspective. This is quite obvious.
Table of Contents:
· End-User in E2E Test Automation means …
∘ 1. 100% separation from the code.
∘ 2. End-users can mostly understand the test scripts
∘ 3. There is no or little playing with the configuration; you only need an endpoint.
∘ 4. End-to-End: from begin and end.
· A bad example: Insurance Claim Testing
· What’s wrong with the approach?
∘ 1. The test script must now be in the same language as the code.
∘ 2. Extra dependencies
∘ 3. Less Control on the test data.
· FAQ
End-User in E2E Test Automation means …
1. 100% separation from the code.
As an end-user of a web app, e.g. Facebook, I test it as I use it. I don’t care whether it is written in PHP or Java.
