> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reliantlabs.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing Workflows

> Validate workflow structure and test runtime behavior before deploying

Reliant provides two levels of workflow testing: **static validation** catches structural problems in your workflow definition, and **scenario testing** verifies runtime behavior with simulated events. Together, they ensure your workflows are both well-formed and behave correctly.

## Static Validation

Reliant validates every workflow when it's loaded. This catches structural problems before your workflow ever runs.

### What Gets Validated

Static validation checks for:

* **Entry point validity**: Every node listed in `entry` must exist in the workflow
* **Node references**: Edges can't reference non-existent node IDs
* **Input completeness**: Every input must have either a `default` value or `required: true`
* **Loop structure**: Loop nodes must have either a `workflow` reference or an `inline` definition
* **Edge consistency**: Edge targets must be valid node IDs within the workflow
* **Reserved names**: Input groups can't use reserved names like `default`
* **CEL expression syntax**: Template expressions and conditions are checked for valid syntax

### Common Validation Errors

| Error                                          | Cause                                             | Fix                                                |
| ---------------------------------------------- | ------------------------------------------------- | -------------------------------------------------- |
| "input X has no default and is not required"   | Input missing both `default` and `required: true` | Add one of the two                                 |
| "node X not found"                             | Edge references a non-existent node ID            | Check spelling of node IDs                         |
| "entry node X not found"                       | `entry` lists a node that doesn't exist           | Match entry names to node IDs                      |
| "loop must have either 'workflow' or 'inline'" | Loop node missing its body                        | Add an `inline` definition or `workflow` reference |

Static validation runs automatically—there's no separate command to invoke it. If your workflow has validation errors, Reliant reports them when loading the workflow and it won't appear in the workflow picker.

You can also trigger validation explicitly using the `validate_workflow` tool in the workflow builder, which is useful when iterating on a workflow definition.

### When Static Validation Isn't Enough

Static validation confirms your workflow is structurally sound, but it can't tell you whether the workflow actually does what you intend. A workflow can pass all validation checks and still route to the wrong node, loop forever, or produce unexpected outputs. That's where scenario testing comes in.

## Scenario Testing

Scenario testing lets you verify workflow runtime behavior without running real LLM calls or tool executions. You define simulated events—mocked LLM responses and tool results—then assert expected outcomes like which nodes were reached, what outputs were produced, and whether the workflow completed successfully.

### How It Works

The scenario simulator executes your workflow's node graph, but instead of calling real activities, it uses your mocked events to produce node outputs. It follows the same edge routing, condition evaluation, and loop logic as the real executor.

### Scenario Structure

A scenario YAML file defines:

```yaml theme={null}
name: my_test_scenario
description: "What this tests"

# Optional: override workflow inputs
inputs:
  max_turns: 5

# Optional: start execution at a specific node (for partial testing)
start_at: execute_tools

# Optional: pre-populate node outputs (for partial testing)
state:
  call_llm:
    tool_calls: [{name: bash, input: {command: ls}}]

# Simulated events (required)
events:
  - type: llm_response
    text: "Hello!"

# Expected outcome (optional but recommended)
expect:
  outcome: completed
  reached: [call_llm, save_result]
```

### Event Types

| Type           | Fields               | Description                           |
| -------------- | -------------------- | ------------------------------------- |
| `llm_response` | `text`, `tool_calls` | Simulates an LLM returning a response |
| `tool_result`  | `tool`, `output`     | Simulates a tool returning results    |
| `tool_error`   | `tool`, `error`      | Simulates a tool failing              |
| `llm_error`    | `error`              | Simulates an LLM API error            |
| `user_input`   | `user_text`          | Simulates user sending a message      |

#### Event Fields

```yaml theme={null}
events:
  # LLM responds with text only (no tool calls)
  - type: llm_response
    text: "I'm done, here's the answer."

  # LLM responds with tool calls
  - type: llm_response
    text: "Let me check that..."
    tool_calls:
      - name: bash
        input: {command: "ls -la"}
      - name: read_file
        input: {path: "/tmp/test.txt"}

  # Tool returns successfully
  - type: tool_result
    tool: bash
    output: {result: "file1.txt\nfile2.txt"}

  # Tool returns an error
  - type: tool_error
    tool: bash
    error: "Permission denied"

  # User sends input
  - type: user_input
    user_text: "Yes, please continue"
```

### Targeting Nodes

By default, events are consumed sequentially — the first triggered node gets the first event, and so on. Use the `node` field to target a specific node:

```yaml theme={null}
events:
  - node: call_llm
    type: llm_response
    text: "Hello!"
  - node: execute_tools
    type: tool_result
    tool: bash
    output: {result: "ok"}
```

Multiple events for the same node are consumed in order each time that node is triggered.

#### Inner Nodes (Loops and Sub-Workflows)

For workflows with inline loops or inline sub-workflows using `type: workflow` with `inline:`, you can target individual nodes inside them using dot-separated qualified IDs:

```yaml theme={null}
events:
  # Target the call_llm node inside agent_loop
  - node: agent_loop.call_llm
    type: llm_response
    tool_calls: [{name: bash, input: {command: ls}}]

  # Target execute_tools inside agent_loop
  - node: agent_loop.execute_tools
    type: tool_result
    tool: bash
    output: {result: "file.txt"}

  # Second iteration: call_llm responds without tools (exits loop)
  - node: agent_loop.call_llm
    type: llm_response
    text: "All done!"
```

The same dot-notation works for inline workflow nodes:

```yaml theme={null}
events:
  # Target the plan node inside a planning sub-workflow
  - node: planning.plan
    output:
      response_text: "Here is the plan."

  # Target the criticize node inside the same sub-workflow
  - node: planning.criticize
    output:
      response_text: "Critique of the plan."
```

For nested structures, chain the IDs:

```yaml theme={null}
events:
  - node: outer_loop.inner_loop.call_llm
    type: llm_response
    text: "Inner step done"
```

#### How inline simulation works

For **inline loops** with an `inline:` definition, the simulator:

1. Creates a sub-state-machine for the inline workflow
2. Executes each inner node individually per iteration
3. Evaluates conditions on inner nodes and skips those with false conditions
4. Evaluates edges within the loop to determine node ordering
5. Calls the mocker with qualified IDs like `loop_id.inner_node_id`
6. Evaluates the `while` condition after each iteration
7. If the sub-workflow declares `outputs:`, evaluates them for the `while` condition

For **inline workflow nodes** with `type: workflow` and `inline:`, the simulator:

1. Creates a sub-state-machine for the inline workflow
2. Evaluates conditions on each inner node and tracks false conditions as skipped
3. Executes inner nodes using qualified IDs like `workflow_id.inner_node_id`
4. Evaluates the sub-workflow's `outputs:` expressions if declared

For **referenced** loops or workflows with `ref:`, the simulator uses black-box mocking with the ref name by default. When the scenario targets qualified inner nodes and the runner can resolve the referenced workflow, it executes that referenced workflow internally instead.

### Expectations

The `expect` section defines assertions:

```yaml theme={null}
expect:
  # Expected outcome
  outcome: completed  # or "error"

  # Nodes that must be executed
  reached:
    - call_llm
    - agent_loop.execute_tools  # inner loop nodes supported
    - planning.plan             # inline workflow inner nodes supported
    - save_result

  # Nodes that must NOT be executed
  not_reached:
    - error_handler
    - agent_loop.fallback_step

  # Nodes that must have been skipped (condition evaluated to false)
  skipped:
    - planning.criticize        # skipped because 'critique' not in steps
    - planning.revise
    - impl_loop.lint            # skipped because 'lint' not in steps

  # Assert specific node output values
  node_outputs:
    call_llm:
      tool_calls: []
    agent_loop.call_llm:  # inner loop nodes supported
      message:
        text: "Done!"

  # Assert error message content
  error_contains: "CEL evaluation"

  # Assert which node produced the error
  error_node: call_llm
```

### Partial Testing

Use `start_at` and `state` to test from a specific point in the workflow:

```yaml theme={null}
name: test_from_execute_tools
start_at: execute_tools

# Pre-populate prior node outputs
state:
  call_llm:
    tool_calls:
      - name: bash
        input: {command: ls}
    message:
      role: assistant
      text: "Let me check"

events:
  - type: tool_result
    tool: bash
    output: {result: "file.txt"}

expect:
  outcome: completed
  reached: [execute_tools, save_result]
  not_reached: [call_llm]  # skipped because we started after it
```

### Loop Testing Patterns

#### Single Iteration (Exit Immediately)

```yaml theme={null}
name: loop_exits_immediately
events:
  - node: agent_loop.call_llm
    type: llm_response
    text: "No tools needed"
expect:
  outcome: completed
  reached: [agent_loop.call_llm, agent_loop.save_result]
  not_reached: [agent_loop.execute_tools]
```

#### Multi-Iteration Loop

Provide events for each iteration. The simulator consumes events per-node sequentially:

```yaml theme={null}
name: loop_three_iterations
events:
  # Iteration 1: calls tool
  - node: agent_loop.call_llm
    type: llm_response
    tool_calls: [{name: bash, input: {command: "echo 1"}}]
  - node: agent_loop.execute_tools
    type: tool_result
    tool: bash
    output: {result: "1"}

  # Iteration 2: calls another tool
  - node: agent_loop.call_llm
    type: llm_response
    tool_calls: [{name: bash, input: {command: "echo 2"}}]
  - node: agent_loop.execute_tools
    type: tool_result
    tool: bash
    output: {result: "2"}

  # Iteration 3: exits loop
  - node: agent_loop.call_llm
    type: llm_response
    text: "All done!"

expect:
  outcome: completed
  reached: [agent_loop, agent_loop.call_llm, agent_loop.execute_tools]
```

#### Testing Loop Output Routing

When a loop's inline workflow declares `outputs:`, those evaluated outputs are available for downstream edge conditions:

```yaml theme={null}
# In the workflow YAML:
# nodes:
#   - id: agent_loop
#     type: loop
#     while: "size(outputs.tool_calls) > 0"
#     inline:
#       outputs:
#         status: "{{nodes.call_llm.message.text}}"
#       ...
# edges:
#   - from: agent_loop
#     cases:
#       - to: success
#         condition: "nodes.agent_loop.status == 'done'"
#     default: fallback

name: loop_routes_to_success
events:
  - node: agent_loop.call_llm
    type: llm_response
    text: "done"
expect:
  reached: [agent_loop, success]
  not_reached: [fallback]
```

### Using Scenario Tools

In the workflow builder, use these tools to manage scenarios:

| Tool              | Usage                                          |
| ----------------- | ---------------------------------------------- |
| `list_scenarios`  | See all scenarios and their status             |
| `view_scenario`   | Examine a scenario's full YAML                 |
| `create_scenario` | Create a new scenario and run it automatically |
| `edit_scenario`   | Make targeted edits to a scenario              |
| `run_scenario`    | Re-run a scenario after workflow changes       |

Scenarios are stored alongside the workflow and run instantly without network calls or LLM API usage.