Cypress Catches Up with Auto-Retry, but in a wrong way

Today (2020–08–21), I read an article on Medium: Flaky Tests: Retrying Failed Tests with Cypress. According to the author: ‘ this was one of the top “must-have” features I’ve been waiting for’. A must-have feature, I totally agree. However, why is it only available now in v5? Did the article's author suggest that Cypress testers had been more or less faking test automation all these years?!

Despite what they claimed, not one Cypress tester I have met could answer it when I asked for a history of test executions of 3 months. The teams performed functional testing most manually, resulting in buggy software and slow releases. There might be a good cypress test engineer somewhere who can create, and more importantly, maintain large user-story level UI tests (e.g. 200 test cases, Level 3 of AgileWay Continuous Testing Grading); In addition, the team needs to really trust the automation suite so that they can push the releases to the production, twice a day. However, I haven’t yet met one that can show reliable daily test executions of 50 cypress tests. Therefore, I remain sceptical.

For people who have known me or read my “Practical Continuous Testing” book, they would know that Auto-Retry has been a BuildWise feature from its first release ( internal: 2013, public: 2017, and won a major international award the next year). The idea of auto-retry came to me in 2007, the time when we solved test scripting maintenance with good test design practices (see my book: Practical Web Test Automation) and TestWise IDE. Soon we had to face the test execution reliability problem: It was difficult to get our 30-min test suite execution 100% pass. Then I came up with the Auto-Retry approach and implemented this feature in CruiseControl via a Java Plugin. When I decided to create my own Continuous Testing server, I was certain that Auto-Retry would be a must-have feature in Iteration 0.

It is good to know that other professionals accept and adopt my auto-retry approach. A few years ago, when my team presented our test automation to a fake chief agile coach, she commented: “ You shall not retry test execution!”. The team kept hard not to laugh until this chief agile coach left. After seeing real Agile, real Continuous Testing and real DevOps, the development team have realised that auto-retry test execution in a Continuous Testing server was obviously a must.

Why is Cypress still wrong?