> For the complete documentation index, see [llms.txt](https://docs.aviator.co/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.aviator.co/mergequeue/concepts/managing-flaky-tests-in-mergequeue.md).

# Optimistic validation

Learn about the flaky test problem in parallel mode and how optimistic validation mitigates queue resets by waiting on later batches, including configuration instructions.

Optimistic validation is a feature to mitigate the queue reset caused by flaky tests in [<mark style="color:blue;">parallel mode</mark>](/mergequeue/concepts/parallel-mode.md). When a test fails in a batch, it waits for other batches to see if a test passes.

## Flaky test problem in parallel mode

In parallel mode, queued PRs are tested in a batch.

<figure><img src="https://273246003-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FOAPqUQVbLbsfI5YESl32%2Fuploads%2FwvwqGhQYoPlh2NjUMJ7V%2FScreen%20Shot%202023-07-16%20at%2011.58.37%20AM.png?alt=media&amp;token=c7440002-43f4-49a4-9b37-de630282f264" alt=""><figcaption><p>Parallel mode with two PRs</p></figcaption></figure>

For example, when there are two PRs (`PR #1` and `PR #2`) enqueued, we create two draft PRs (`Draft PR #3` and `Draft PR #4`) for testing. These two draft PRs are based on the latest `main` branch and run tests to check tests won't fail after merge. `Draft PR #3` includes `PR #1` and `Draft PR #4` includes `PR #1` and `PR #2`. This way, the tests of these PRs can run in parallel, making sure that the post-merge state still pass the tests.

When a CI run fails in one of those parallel build draft PRs, that failing a PR gets dequeued. This causes a queue reset. Imagine that `Draft PR #3` fails the CI run. This includes `PR #1` and this PR gets dequeued. `Draft PR #4` also includes `PR #1` and the CI run for this PR is cancelled. Because all the PRs created after `PR #1` includes it, this makes a cascading effect and effectively the entire queue is reset.

If your test is flaky, this queue reset happens frequently. This slows down the entire queue and affects the efficacy of the team.

{% hint style="info" %}
This page covers optimistic validation, which waits on later batches without reading the failure. [Flaky test management](/mergequeue/concepts/flaky-test-management.md) (beta) goes further: it reads the failure, reruns the ones that would pass on a retry, and stops those failures counting against the depth configured here.
{% endhint %}

## Optimistic validation

Optimistic validation is a feature to address this situation. When a CI run fails for a draft PR, instead of immediately dequeue the PR and causing a queue reset, it waits for other PRs to finish.

Continue from the example above. When a CI run for `Draft PR #3` fails, without optimistic validation, it dequeues the `PR #1` and resets the entire queue. With optimistic validation, it waits for other draft PRs (in this case `Draft PR #4`) to finish the CI runs. When they pass the CI, it assumes that the CI run would have been passed and merges the PRs.

## Configuring optimistic validation

There are two settings to configure optimistic validation. Both are specified within the parallel mode section of the configuration.

`use_optimistic_validation` - boolean. When the top draft PR hasn't passed (still running, or failed), Aviator checks the draft PRs below it, and if any of them has passed, it treats the top as passing and merges it. Defaults to `true`.

`optimistic_validation_failure_depth` - integer, default `1`. How many failed draft PRs, counting the top one, Aviator waits for before dequeuing a failed top PR, as long as some draft PRs below it are still running. At `1`, a failed top PR is dequeued unless a draft PR below it has already passed. Requires `use_optimistic_validation` to be enabled.

```
merge_rules:
  labels:
    trigger: "mergequeue"
  merge_mode:
    type: "parallel"
  parallel_mode:
    max_parallel_builds: 10
    use_optimistic_validation: true
    optimistic_validation_failure_depth: 2
```

## Side-effect of optimistic validation

By introducing optimistic validation, real errors will take longer to be kicked out of the queue. This is because when there is a CI run failure, it keeps waiting for other batches to finish. Therefore, the failure depth should generally be kept small.

Most of the Aviator users find this side-effect is acceptable as the number of real failures in the queue are relatively small. However, this depends on how flaky your tests are.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.aviator.co/mergequeue/concepts/managing-flaky-tests-in-mergequeue.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.