Which is a common root cause of flaky tests?
Options
- Using too many assertions in one single test
- Depending on time, shared state or real network (Correct option)
- Using fixtures to prepare the data for tests
- Running the tests on a local developer machine
Answer
Dependence on time, ordering, shared state, randomness or real network calls
Why
A flaky test sometimes passes and sometimes fails without any code change. Teams then stop trusting the test suite.
Typical causes: depending on the current time, random values, test order, data left by other tests, real network calls, or sleep() used to wait for something.
Fix the cause (inject the clock, seed randomness, isolate data, mock the network, wait for conditions). Automatic retries only hide it.
Remember
Flaky = non-deterministic. Remove the source of randomness.