Member-only story
Execution Speed of Automated End-to-End (via UI) Testing Clarified
Test Execution Speed is just one factor of many in Test Automation.
This article is one of the “IT Terminology Clarified” series.
Execution Speed is an interesting subject in Test Automation, here, I mean, end-to-end via UI. For example, web test automation, the most common form of test automation, drives the app in a Chrome browser for testing purposes.
Table of Contents
· The Cypress Lie on Test Execution Speed
· From an overall perspective in test automation, speed is minor compared to other factors
· Speed is important in the developing/debugging phase.
· Speed is not that important in the daily execution phase.
The Cypress Lie on Test Execution Speed
Since ~2018, Cypress’ marketing team has lured many fake automated testers using a big, fat lie. (Check out this article)
I don’t know how Cypress got this conclusion. It is against my observation, besides every Cypress test automation attempt (I witnessed) failed, in the process of rescuing some of them (to use raw Selenium WebDriver), I found Cypress was slower than raw Selenium. Maybe the people before me did not develop selenium tests properly. Check out my article: Optimize Selenium WebDriver Automated Test Scripts: Speed.
Some Cypress defenders would argue otherwise, saying “blah, blah”, but no one showed me a solid and convincing result yet. Yes, there might be cases Cypress was a bit faster on a particular website (due to some optimization, we all know that trick).
As a software testing company, Cypress shall at least show the execution results and the test suite to back its claim, right? None. Just a statement followed by marketing words. Sadly, so many fake automated testers fell for it. If some of them stick with the right track, raw Selenium WebDriver, they might grow and become a real test automation engineer.
There is more than one benchmark test to show the opposite of Cypress’ claim on speed (actually slower, not ‘much, much faster’), such as:
