mirror of
https://github.com/servo/servo.git
synced 2025-10-04 02:29:12 +01:00
5e30ce8532
There are two kinds of flaky/intermittent tests in Servo. The traditional kind is the test that fails on the CI, but has an associated bug indicating that the test is an intermittent failure. Many of these tests have completely unstable results, for instance those where an unpredictable set of subtests fail. It's impossible to generate stable results for these, so we have traditionally simply discard these unexpected results. Another kind of intermittent test is one that will produce an expected result when rerun (ie will flake). Some of these are also labeled with bugs, while some are not. In some cases, there is flakiness in some core Servo functionality that can lead to *any* test flaking, such as a race condition that can lead to an early screenshot for reftests. When these kinds of tests do not have associated bugs, they cause the CI to fail. In this case, it is impossible to label these tests as intermittent because it can literally be any test. This change, reruns failed tests in order to detect unlabeled tests in the second category. Instead of blocking the CI when the second run leads to expected results, the CI will now pass, but the flake will be reported to the new flakiness dashboard. This prevents unrelated flakes from slowing down the merge queue. |
||
|---|---|---|
| .. | ||
| fontconfig | ||
| performance | ||
| upstream-wpt-changes | ||
| buildbot_steps.yml | ||
| chaos_monkey_test.py | ||
| check_dynamic_symbols.py | ||
| check_no_panic.sh | ||
| clean_build_artifacts.sh | ||
| generate_workflow.py | ||
| lockfile_changed.sh | ||
| macos_package_smoketest.sh | ||
| mutation_test.sh | ||
| report_aggregated_expected_results.py | ||
| update-wpt-checkout | ||
| wpt-nightly-update.sh | ||
| xorg.conf | ||