Flaky Tests in E2E Test Automation Clarified — Part A: Flaky Tests are Common but People tend to Neglect them

Before addressing the flaky tests, we need to treat them honestly and seriously first.

In this series:

• Part A: Flaky Tests are Common but People tend to Neglect them
• Part B: Test Suite Size Matters *
• Part C: Causes 1–3 (upcoming)
• Part D: Causes 4–7 (upcoming)
• Part E: Solutions 1–3 (upcoming)
• Part F: Solutions 4 -7* (upcoming)
• Part G: FAQ (upcoming)

In 2018, I watched the best presentation on CI/CD, “Continuous Integration at Facebook”. The presenter showed three characteristics of CI at Facebook:

The first item, “High-Signal”, means ideally the automated test failures are mostly genuine, not false positives which are test failures due to non-application reasons. A common cause for false positives is ‘flaky tests’.