> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reliantlabs.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Test AI agent workflows with scenarios before you ship

> Test every path of an AI agent workflow before it runs for real: mock agents and commands in scenarios, run them locally, and read the mismatch when one fails.

By the end of this guide a scenario covers your workflow's success path, its failure path and one edge case, and one command checks them all in seconds, with no tokens spent and no command run.

A scenario runs your workflow on the real workflow engine, in memory. Only the activities are mocked: agent results, command output and integration calls come from the scenario file. Edges, CEL conditions, loops, messages and `on_failure` run as they would in production ([How workflows run](https://docs.reliantlabs.io/deep-dives/how-workflows-run.md)).

## Before you begin

* The `reliant` CLI, v1.21 or later (`reliant version`). Scenarios need no account and no network.
* The basics of a workflow file, from [Write your first multi-agent workflow](https://docs.reliantlabs.io/guides/workflows/first-multi-agent-workflow.md).

**1. Start from a workflow**

This workflow runs the test suite every night. When the tests fail, an agent reads the output and says why.

```yaml .reliant/workflows/nightly-tests.yaml theme={null}
name: nightly-tests
title: Nightly tests
description: Run the test suite every night. When it fails, an agent reads the output and says why.

inputs:
  test_command:
    type: string
    default: go test ./...
    description: The command that must exit 0

triggers:
  - name: nightly
    schedule:
      cron: ["0 2 * * *"]
      timezone: UTC

entry: [test]

nodes:
  - id: test
    type: run
    command: "{{inputs.test_command}}"
    timeout: 15m

  - id: diagnose
    type: workflow
    ref: builtin://agent
    args:
      mode: plan
    thread:
      mode: fork
      inject:
        role: user
        content: |
          `{{inputs.test_command}}` failed with exit code {{nodes.test.exit_code}}.
          Read the output and the code, and say in three sentences what broke and where.
          Do not change any files.

          {{nodes.test.stdout + nodes.test.stderr}}

  - id: green
    type: save_message
    outcome: success
    args:
      role: assistant
      content: "Nightly tests pass."

  - id: red
    type: save_message
    outcome: failure
    args:
      role: assistant
      display_style: warning
      content: |
        Nightly tests fail (exit {{nodes.test.exit_code}}).
        {{nodes.diagnose.response_text}}

edges:
  - from: test
    cases:
      - to: green
        condition: nodes.test.exit_code == 0
    default: diagnose
  - from: diagnose
    default: red

on_failure:
  entry: [report]
  nodes:
    - id: report
      type: save_message
      args:
        role: assistant
        display_style: warning
        content: "nightly-tests stopped at {{failure.node_id}}: {{failure.error}}"
```

`reliant workflow lint` passes it with one warning: `[scenario-none] the workflow has no scenarios, so none of its triggers or endings is tested`.

**2. Cover the success path**

Scenarios live in `.reliant/workflows/<name>/scenarios/`, one per file, where `<name>` is the workflow's `name:`. Each event mocks one node by its path. `trigger:` starts the run the way the `nightly` schedule would:

```yaml .reliant/workflows/nightly-tests/scenarios/green_overnight.yaml theme={null}
name: green_overnight
description: The 2am firing runs the suite, it passes, and no agent is started

trigger:
  name: nightly

events:
  - node: test
    output: {exit_code: 0, stdout: "ok  example.com/app  0.41s\n"}

expect:
  outcome: completed
  completed: [test, green]
  not_reached: [diagnose, red]
  messages:
    green:
      content_contains: ["Nightly tests pass."]
```

A `run` node's mock is its whole output: `exit_code`, `stdout`, `stderr`. A `ref:` node such as `diagnose` is mocked as a unit by an event on the node itself.

**3. Cover the failure path**

With only the first scenario, lint names the ending nothing tests yet:

```text theme={null}
Lint warning: [scenario-terminal-uncovered] red: no scenario expects a run to end here (add a scenario whose expect.completed (or reached) names red)
```

```yaml .reliant/workflows/nightly-tests/scenarios/red_tests_get_a_diagnosis.yaml theme={null}
name: red_tests_get_a_diagnosis
description: The suite fails, the agent explains it, and the run ends failed

events:
  - node: test
    output: {exit_code: 1, stdout: "--- FAIL: TestSlug (0.00s)\n    slug_test.go:12: got \"Hello-World\", want \"hello-world\"\n"}
  - node: diagnose
    output: {response_text: "TestSlug fails because Slug no longer lowercases its input."}

expect:
  outcome: failed
  completed: [test, diagnose, red]
  not_reached: [green]
  messages:
    red:
      content_contains: ["exit 1", "no longer lowercases"]
  on_failure:
    ran: true
    node_id: red
```

`outcome: failed` means the run ended at a node that declares `outcome: failure`. `on_failure` checks that the handler ran and which node it saw.

> [!WARNING]
>
> Assert `outcome: failed` for a failure ending. `outcome: completed` also matches a failed run, and `scenario run` reports it as passed without a warning.

**4. Run the scenarios**

```bash terminal theme={null}
reliant workflow scenario run
```

```text theme={null}
Running 2 scenario(s) across 1 workflow(s)

  ✓ nightly-tests/green_overnight (443ms)
  ✓ nightly-tests/red_tests_get_a_diagnosis (7ms)
...
✅ All scenarios passed
```

`reliant workflow lint` now reports `0 lint error(s), 0 warning(s)`: every trigger and every ending is covered.

**5. Write the next expectation first**

When the test tool is not installed, the command exits `127`, and the workflow pays an agent to read `command not found`. Write the behavior you want before you change the workflow:

```yaml .reliant/workflows/nightly-tests/scenarios/missing_tool_skips_the_agent.yaml theme={null}
name: missing_tool_skips_the_agent
description: The test command is not installed, so no agent is paid to read "command not found"

inputs:
  test_command: gotestsum ./...

events:
  - node: test
    output: {exit_code: 127, stderr: "bash: gotestsum: command not found\n"}

expect:
  outcome: failed
  completed: [test]
  not_reached: [diagnose]
```

```bash terminal theme={null}
reliant workflow scenario run --filter missing_tool --verbose
```

```text theme={null}
  ✗ nightly-tests/missing_tool_skips_the_agent (453ms)
      Mismatch: expected outcome "failed" but got "error" (node "red": CEL evaluation failed for step red: ... CEL evaluation error: no such key: response_text ...)
      Mismatch: expected node "diagnose" NOT to be reached but it was
...
❌ Scenarios failed
```

The second mismatch is the one you wrote. The first shows what an unmocked node does: `diagnose` has no event in this scenario, so it returned an empty output, and `red` could not read `response_text`. Without `--verbose`, only the first mismatch prints, followed by `... and 1 more`.

**6. Fix the workflow and run again**

Add a node for the missing tool and a case that routes exit `127` to it:

```yaml .reliant/workflows/nightly-tests.yaml theme={null}
nodes:
  - id: misconfigured
    type: save_message
    outcome: failure
    args:
      role: assistant
      display_style: warning
      content: "`{{inputs.test_command}}` is not installed on this machine: {{nodes.test.stderr}}"

edges:
  - from: test
    cases:
      - to: green
        condition: nodes.test.exit_code == 0
      - to: misconfigured
        condition: nodes.test.exit_code == 127
    default: diagnose
```

Then change the scenario's expectation to `completed: [test, misconfigured]`, so it also covers the new ending.

**7. Verify the whole suite**

```bash terminal theme={null}
reliant workflow scenario run
```

```text theme={null}
Running 3 scenario(s) across 1 workflow(s)

  ✓ nightly-tests/green_overnight (450ms)
  ✓ nightly-tests/missing_tool_skips_the_agent (6ms)
  ✓ nightly-tests/red_tests_get_a_diagnosis (8ms)
...
✅ All scenarios passed
```

The command exits `0` when every scenario passes and `1` when any fails, so it can gate a commit or a CI job. `--json` prints one result per scenario, with its `mismatches`. To narrow a run, pass a workflow file (`reliant workflow scenario run .reliant/workflows/nightly-tests.yaml`) or `--filter <text>`, which matches scenario names. `--fail-fast` stops at the first failure. `reliant workflow scenario list` shows what would run.

## Learn from the built-in workflows

In reliant v1.21.3 the built-in workflows ship 217 scenarios, with mocked approvals, GitHub actions, loops and triggers. `--include-builtins` adds them to `scenario list` and `scenario run`. The [scenario format](https://docs.reliantlabs.io/workflows/testing.md) and the [scenario schema](https://docs.reliantlabs.io/reference/scenario-schema.md) document every field.

## What this does not do

* **It never runs anything real.** A passing scenario proves the routing, the CEL and the messages. It does not prove the agent can find the bug or that the command exists on your machine. Run the workflow once for real before you put it on a schedule.
* **It does not check what only a live run enforces.** For example, an `ask_question` node without `"type":"ask_user"` in its metadata passes `validate`, `lint` and every scenario, but the live question shows the person no options.
* **It trusts your mocks.** A node you do not mock returns an empty output. An output key you misspell is ignored: a `run` mock of `{exitcode: 1}` is read as exit `0` and takes the green path, and `scenario run` prints no warning. Copy output field names from the [nodes reference](https://docs.reliantlabs.io/reference/nodes.md).

## Troubleshooting

> [accordion] validation failed: event\[1]: unknown node "diagnos"
>
> The event names a node the workflow does not have. The message lists the valid paths, for example `(valid nodes: diagnose, green, misconfigured, on_failure.report, red, test)`. Nodes inside a loop or an inline workflow are `<parent>.<id>`. The scenario counts under `Errors`, not `Failed`.

> [accordion] validation failed: expect.completed: unknown node
>
> An expectation names a node that does not exist yet. Add the node to the workflow first, or leave it out of the expectation until you do.

> [accordion] CEL evaluation error: no such key: response\_text
>
> A later node reads an output that a node you did not mock never produced. Add an event for that node, such as `diagnose` with `output: {response_text: "..."}`.

## Next steps

> [card] [Catch workflows that break unattended](https://docs.reliantlabs.io/guides/workflows/lint-unattended-workflows.md)
>
> Lint rules for timeouts, unbounded loops and concurrency slots.

> [card] [Loop an agent until tests pass](https://docs.reliantlabs.io/guides/workflows/run-tests-until-they-pass.md)
>
> Scenarios for a loop that fails twice and then passes.

> [card] [Require human approval](https://docs.reliantlabs.io/guides/workflows/require-human-approval.md)
>
> Mock an approval as approved, denied or timed out.

> [card] [How workflows run](https://docs.reliantlabs.io/deep-dives/how-workflows-run.md)
>
> The engine scenarios run on, and what is mocked.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.