---
title: "Flaky test detection and quarantine"
description: "CoderHelm reads the end to end results your CI produces, tells flaky tests from broken ones, keeps flaky tests from failing builds and opens fix pull requests that must pass twenty runs in a row."
url: https://coderhelm.com/tests/
updated: 2026-11-05
---

# Flaky tests stop blocking you. Broken ones never hide.

> CoderHelm reads every end to end run your CI already produces, tells a flaky test from a broken one, and acts on it.

*Product view: The Tests page: each test's history run by run, flaky and broken tests told apart, and the quarantine list.*

## Flaky, broken or a bad run

- **Told apart:** A test that passed and failed on the same commit is flaky. One that fails almost every time is broken. A run where many tests fail together is a bad run, not many flaky tests.
- **"Is this failure mine?":** Each failing test on a pull request gets one line in the review: known flaky with its history, a bad run, or likely caused by the files this pull request changed.
- **Quarantine, never for broken tests:** When your team turns it on, flaky tests keep running and being recorded but stop failing the build. A broken test is never quarantined. Owners get a ticket after two days by default; each team picks between 1 and 30.
- **Quarantined failures don't block:** When the only failing tests in a run are quarantined ones, CoderHelm treats the run as passing: it does not start a CI fix, armed auto merge goes ahead, and a note on the pull request names the tests.
- **Fix pull requests with proof:** CoderHelm reproduces the flake, fixes it, and marks the pull request ready only after twenty green runs in a row.
- **Tests you are missing:** Uncovered changes and bug fixes without tests become suggested tests, each proposed only after it passes ten repeat runs.
- **Every run on one page:** Runs, each test's history, failures grouped by cause, the slowest tests and the quarantine list.
- **Fits your CI:** Quarantined tests still run in your CI. The Tests page shows the workflow step that keeps them from failing the build itself.

## Reports it reads

Playwright (JSON or HTML), JUnit XML and Cucumber JSON. Results that never reach GitHub, such as a device farm, can be uploaded with a GitHub Actions token, so there is no secret to store.

## Related

- [AI that fixes failing CI and answers review comments](https://coderhelm.com/maintenance.md): CoderHelm stays on the code after the pull request: it answers review comments, fixes failing CI on its pull requests, resolves merge conflicts, checks deploys after merge and writes release notes.
- [AI code review for GitHub pull requests](https://coderhelm.com/reviewer.md): Add a label to any GitHub pull request and CoderHelm reviews it with the context of the whole repository, comments on exact lines with suggested fixes, rates risk and reviews again on every push.
- [GitHub issue and Jira ticket to pull request](https://coderhelm.com/pull-requests.md): Assign a GitHub issue or Jira ticket to CoderHelm and it plans the change, writes it on its own branch, runs your build and tests, reviews its own work and opens a pull request.

---

Source: https://coderhelm.com/tests/ · Markdown: https://coderhelm.com/tests.md · Updated 2026-11-05 · Get started: https://app.coderhelm.com