Why Cypress Sucks for Real Test Automation?
--
This article is one of the “Be aware of Fake Test Automation/DevOps Engineers” series.
Note: The sequel to this article “Why Cypress Sucks for Real Test Automation? (Part 2: Limitations)” is now available.(2021–02–15)
Update (2021–04): “Why JavaScript Is Not a Suitable Language for Real Web Test Automation?”.
Update (2022–06–22): “Cypress vs Selenium WebDriver Comparison by Example”
Update (2023–08): “Cypress.io is Dying”
Cypress, a proprietary JavaScript end-to-end testing tool, seems to me that it is an improved Selenium v1 (also based on JavaScript), which was a big hype in 2005 (until it was merged with WebDriver in 2009). I believe many testers had not used Selenium v1 before, what I could tell you is that being replaced by Selenium WebDriver (i.e. Selenium 2) was a good thing.
As a test automation & continuous testing coach, I visited many software projects. I have never seen a single success using Cypress, not even Level 2 of AgileWay Continuous Testing grading (in ‘Practical Continuous Testing’ book), that is:
- A regression suite of 50+ user-story level automated UI tests
- The test suite runs in a CI (or CT) server multiple times a day as regression testing. (otherwise, there was no point doing it)
- while coping with frequent application changes (reality, right?)
- At least one green build (passing all tests, or with explainable causes) a day.
- On a green build, the team has high degrees of confidence to release to the production. (that’s the ultimate purpose of doing test automation, isn’t it?)
How do you score? As you can imagine, Level 3 (200+ tests) will be much much harder. All successful test automation implementations I have seen (up to Level 4, 500+ automated UI tests run in CT daily) were all using Selenium WebDriver/Watir + RSpec. I heard that Facebook's test suite is huge, if you are interested in large-scale test automation, check out this F8 presentation by a Facebook engineer. By the way, Facebook uses Selenium WebDriver.
