Case Study: Rescue Unreliable 20 hours of Automated Regression Testing in Jenkins ⇒ 6-Minute Highly-Reliable in BuildWise CT Server
Convert unreliable tests in self-created Java/Groovy ‘framework’ to raw Selenium WebDriver in Ruby; Switch Jenkins to BuildWise to execute the test suite.
The title might sound unbelievable, but this is a true story, there is a solid proof towards the end of the article.
About nine years ago, I joined a large tech company as a test automation engineer (contractor). On the first day, the division test automation lead expressed that he wanted me to rescue test automation in a high-priority project. He said: “We (including him) designed the test automation framework, but the tester in the team couldn’t develop and maintain the test scripts”. Hearing that, I had known a part of the problem for the project already. (Check out my article: Please, Not Another Web Test Automation Framework, Just Use Raw Selenium WebDriver).
Table of Contents:
· The Test Automation and CI/CD Mess
· The First Day
· The Rescue Proposal
· The Rescue
∘ Stage 1: Convert the key test cases to raw Selenium
∘ Stage 2: Run these days daily in the BuildWise CT server
· The Result
The Test Automation and CI/CD Mess
The test automation lead introduced me to the team. I was happy to know that I used to work with the project manager (M) not long ago and the business analyst (U) many years ago. The project manager knew my capability in test automation well, and he said this in the project review meeting: “In my 30 years in the software industry, I have seen no one even close to what Zhimin achieved in test automation”. The business analyst was surprised that I came in as a tester. From her memory, I was a contract senior C# developer then (check out Why I Switched my day work from Programming to Automated Testing?). In this team, there were also two software engineers and one dedicated tester.
After the casual conversation with the team, I started to ask about the current status of testing. Apparently, it didn’t go well, and the test automation was about to be abandoned. In fact, most of the…