Lessons Learned from Test Failures due to a new ChromeDriver Bug

On 2019–06–15, I upgraded my test lab with BuildWise Agent v1.5.1 (including the latest ChromeDriver 75.0.3770.8) and triggered one of my continuous testing process: executing 603 Selenium WebDriver automated regression tests for ClinicWise. The build failed!

The build log showed the failure: the downloaded file was not at the expected location.

I was quite sure that no code changes could cause the issue, since the last successful build. Still, I re-ran the failed tests in TestWise IDE and verified that the file was not saved to the specified folder.

@download_path = File.expand_path File.join(File.dirname(__FILE__), "..", "tmp")
prefs = {
        :download => {
            :prompt_for_download => false,
            :default_directory => @download_path
        }
 }
@driver = Selenium::WebDriver.for :chrome, :prefs => prefs

From the build history on the CT server, I was pretty sure the cause was neither code nor test scripts. It most likely was: Chrome browser (v75) or ChromeDriver v75. I checked ChromeDriver’s release log, no mentioning changes on the chrome preferences there. Then I visited ChromeDriver Google Group, found one related post:

A user: “ Is chrome following your preferences for this? Currently, my default_directory is not being followed and downloads are landing in the users/CURRENT_USER/downloads directory.”

Google Dev: “This is a known bug in the current release of ChromeDriver 75. Please see https://crbug.com/chromedriver/2943 for more details. We expect to have an update to ChromeDriver 75 tomorrow with the fix. Sorry about the inconvenience.”

Unless I found out an alternative syntax with Chome preferences, I would just have to wait for the next releases of Chrome browser and/or ChromeDriver. But in the meantime, I needed to push the build to production, as Daily Deployment (an XP practice) has become my habit. After verifying all test failures (on BuildWise run) were the same reason, I added a note to the build.