Testing Workflows

Reliant provides two levels of workflow testing: static validation catches structural problems in your workflow definition, and scenario testing verifies runtime behavior with mocked events. Together, they ensure your workflows are both well-formed and behave correctly.

Static Validation

Reliant validates every workflow when it’s loaded. This catches structural problems before your workflow ever runs.

What Gets Validated

Static validation checks for:

Entry point validity: Every node listed in entry must exist in the workflow
Node references: Edges can’t reference non-existent node IDs
Input completeness: Every input must have either a default value or required: true
Loop structure: Loop nodes must have either a workflow reference or an inline definition
Edge consistency: Edge targets must be valid node IDs within the workflow
Reserved names: Input groups can’t use reserved names like default
CEL expression syntax: Template expressions and conditions are checked for valid syntax

Common Validation Errors

Error	Cause	Fix
”input X has no default and is not required”	Input missing both `default` and `required: true`	Add one of the two
”node X not found”	Edge references a non-existent node ID	Check spelling of node IDs
”entry node X not found”	`entry` lists a node that doesn’t exist	Match entry names to node IDs
”loop must have either ‘workflow’ or ‘inline‘“	Loop node missing its body	Add an `inline` definition or `workflow` reference

Validation runs whenever a workflow is loaded or saved. A project workflow file with errors is listed in Workflows → Library under Failed to load, with the error, and is not offered in the workflow picker; a workflow in the builder with errors cannot be published until they are fixed. The workflow tools an agent uses (create_workflow, edit_workflow, write_workflow) validate every write and return the problems, and get_workflow reports whether a workflow is valid. To check a file yourself, run reliant workflow validate path/to/workflow.yaml. It follows every ref: transitively, exits non-zero on any error, and with --trigger-payload dry-runs the workflow’s triggers against a sample event (see Automations).

When Static Validation Isn’t Enough

Static validation confirms your workflow is structurally sound, but it can’t tell you whether the workflow actually does what you intend. A workflow can pass all validation checks and still route to the wrong node, loop forever, or produce unexpected outputs. That’s where scenario testing comes in.

Scenario Testing

Scenario testing lets you verify workflow runtime behavior without running real LLM calls or tool executions. You define simulated events—mocked LLM responses and tool results—then assert expected outcomes like which nodes were reached, what outputs were produced, and whether the workflow completed successfully.

How It Works

Scenarios run on the real workflow runtime — the same engine that executes your workflow in a chat — inside an in-memory environment. Only the activity layer (LLM calls, tool execution, worktree and daemon operations) is replaced: each activity’s result comes from your mocked events. Edge routing, conditions, loops, joins, sub-workflows, spawns, CEL evaluation and save_message resolution are all production code, so a passing scenario proves the workflow actually runs. Any activity a scenario cannot mock fails the run by name; nothing ever executes for real.

Scenario Structure

A scenario YAML file defines:

name: my_test_scenario
description: "What this tests"

# Optional: override workflow inputs
inputs:
  max_turns: 5

# Optional: start execution at a specific node (for partial testing)
start_at: execute_tools

# Optional: pre-populate node outputs (for partial testing)
state:
  call_llm:
    tool_calls: [{name: shell, input: '{"command": "ls"}'}]

# Simulated events (required)
events:
  - type: llm_response
    text: "Hello!"

# Expected outcome (optional but recommended)
expect:
  outcome: completed
  reached: [call_llm, save_result]

Event Types

Type	Fields	Description
`llm_response`	`text`, `tool_calls`	Simulates a Call LLM node returning a response
`tool_result`	`tool`, `tool_output` (or `output`), `is_error`	Simulates an Execute Tools node returning one tool’s result. `is_error: true` makes it a failed call
(no type)	`output`	Any other node: `output` is the node’s whole output, such as an approval’s `status`, an ask_question’s `response` or a run node’s `exit_code`

An event without a type must look like the real node’s output: it is checked against that node type’s output schema, so a mock cannot claim a shape the real node never produces.

Event Fields

events:
  # LLM responds with text only (no tool calls)
  - type: llm_response
    text: "I'm done, here's the answer."

  # LLM responds with tool calls
  - type: llm_response
    text: "Let me check that..."
    tool_calls:
      - name: shell
        input: {command: "ls -la"}
      - name: view
        input: {file_path: "/tmp/test.txt"}

  # Tool returns successfully
  - type: tool_result
    tool: shell
    tool_output: {result: "file1.txt\nfile2.txt"}

  # Tool call fails
  - type: tool_result
    tool: shell
    tool_output: {error: "Permission denied"}
    is_error: true

  # Any other node: its raw output
  - node: confirm
    output: {status: approved}

Targeting Nodes

By default, events are consumed sequentially — the first triggered node gets the first event, and so on. Use the node field to target a specific node:

events:
  - node: call_llm
    type: llm_response
    text: "Hello!"
  - node: execute_tools
    type: tool_result
    tool: shell
    output: {result: "ok"}

Multiple events for the same node are consumed in order each time that node is triggered.

Inner Nodes (Loops and Sub-Workflows)

For workflows with inline loops or inline sub-workflows using type: workflow with inline:, you can target individual nodes inside them using dot-separated qualified IDs:

events:
  # Target the call_llm node inside agent_loop
  - node: agent_loop.call_llm
    type: llm_response
    tool_calls: [{name: shell, input: '{"command": "ls"}'}]

  # Target execute_tools inside agent_loop
  - node: agent_loop.execute_tools
    type: tool_result
    tool: shell
    output: {result: "file.txt"}

  # Second iteration: call_llm responds without tools (exits loop)
  - node: agent_loop.call_llm
    type: llm_response
    text: "All done!"

The same dot-notation works for inline workflow nodes:

events:
  # Target the plan node inside a planning sub-workflow
  - node: planning.plan
    output:
      response_text: "Here is the plan."

  # Target the criticize node inside the same sub-workflow
  - node: planning.criticize
    output:
      response_text: "Critique of the plan."

For nested structures, chain the IDs:

events:
  - node: outer_loop.inner_loop.call_llm
    type: llm_response
    text: "Inner step done"

How inner nodes execute

Inline loops and inline workflow nodes always execute their bodies. Each inner node’s mock is looked up by its qualified ID (loop_id.inner_node_id, workflow_id.inner_node_id), inner conditions skip nodes exactly as they do at runtime, and a loop’s while and declared outputs: are evaluated after each iteration. Referenced loops or workflows (ref:) are black-boxed by default: an event on the node itself (node: review) supplies the node’s whole output, and the referenced workflow’s body does not run. When the scenario targets qualified inner nodes (node: review.agent_loop.call_llm), the referenced workflow is loaded and executes for real instead. To mock a loop as a unit — its body does not run — set black_box: true on its event; the event’s output is the loop’s aggregate output (_results, _iterations, declared outputs) exactly as downstream nodes read it. Events are always keyed by node path. Keying an event by a workflow ref (node: builtin://agent) is a validation error: one ref can back many nodes, so name the node that references it.

Expectations

The expect section defines assertions:

expect:
  # Expected outcome
  outcome: completed  # or "error", or "failed"

  # Nodes that must have been scheduled: run, skipped or errored.
  # This does not prove a node ran; use completed for that.
  reached:
    - call_llm
    - agent_loop.execute_tools  # inner loop nodes supported
    - planning.plan             # inline workflow inner nodes supported
    - save_result

  # Nodes that must have run to completion
  completed:
    - save_result

  # Nodes that must NOT be scheduled
  not_reached:
    - error_handler
    - agent_loop.fallback_step

  # Nodes that must have been skipped (condition evaluated to false)
  skipped:
    - planning.criticize        # skipped because 'critique' not in steps
    - planning.revise
    - impl_loop.lint            # skipped because 'lint' not in steps

  # Assert specific node output values
  node_outputs:
    call_llm:
      tool_calls: []
    agent_loop.call_llm:  # inner loop nodes supported
      message:
        text: "Done!"

  # Assert the workflow's declared outputs
  outputs:
    summary: "Done!"

  # Assert the messages a node saved to the thread
  messages:
    save_result:
      saved: true
      role: assistant
      content_contains: ["Done"]

  # Assert error message content
  error_contains: "CEL evaluation"

  # Assert which node produced the error
  error_node: call_llm

A node’s messages come from its save_message: block, or from the node itself for a save_message node, resolved by the real runtime against each mocked result. saved: false asserts it saved nothing; count asserts an exact number (a node in a loop saves once per iteration); content_not_contains rules text out.

Partial Testing

Use start_at and state to test from a specific point in the workflow:

name: test_from_execute_tools
start_at: execute_tools

# Pre-populate prior node outputs
state:
  call_llm:
    tool_calls:
      - name: shell
        input: {command: ls}
    message:
      role: assistant
      text: "Let me check"

events:
  - type: tool_result
    tool: shell
    output: {result: "file.txt"}

expect:
  outcome: completed
  reached: [execute_tools, save_result]
  not_reached: [call_llm]  # skipped because we started after it

Loop Testing Patterns

Single Iteration (Exit Immediately)

name: loop_exits_immediately
events:
  - node: agent_loop.call_llm
    type: llm_response
    text: "No tools needed"
expect:
  outcome: completed
  reached: [agent_loop.call_llm, agent_loop.save_result]
  not_reached: [agent_loop.execute_tools]

Multi-Iteration Loop

Provide events for each iteration. Events for the same node are consumed in order, one per execution:

name: loop_three_iterations
events:
  # Iteration 1: calls tool
  - node: agent_loop.call_llm
    type: llm_response
    tool_calls: [{name: shell, input: {command: "echo 1"}}]
  - node: agent_loop.execute_tools
    type: tool_result
    tool: shell
    output: {result: "1"}

  # Iteration 2: calls another tool
  - node: agent_loop.call_llm
    type: llm_response
    tool_calls: [{name: shell, input: {command: "echo 2"}}]
  - node: agent_loop.execute_tools
    type: tool_result
    tool: shell
    output: {result: "2"}

  # Iteration 3: exits loop
  - node: agent_loop.call_llm
    type: llm_response
    text: "All done!"

expect:
  outcome: completed
  reached: [agent_loop, agent_loop.call_llm, agent_loop.execute_tools]

Testing Loop Output Routing

When a loop’s inline workflow declares outputs:, those evaluated outputs are available for downstream edge conditions:

# In the workflow YAML:
# nodes:
#   - id: agent_loop
#     type: loop
#     while: "size(outputs.tool_calls) > 0"
#     inline:
#       outputs:
#         status: "{{nodes.call_llm.message.text}}"
#       ...
# edges:
#   - from: agent_loop
#     cases:
#       - to: success
#         condition: "nodes.agent_loop.status == 'done'"
#     default: fallback

name: loop_routes_to_success
events:
  - node: agent_loop.call_llm
    type: llm_response
    text: "done"
expect:
  reached: [agent_loop, success]
  not_reached: [fallback]

Using Scenario Tools

An agent manages scenarios with these workflow tools (in any chat, load_tool(name="tag:workflow") loads them):

Tool	Usage
`list_scenarios`	See all scenarios and their status
`view_scenario`	Examine a scenario’s full YAML
`write_scenario`	Create a scenario, or replace one, from YAML
`edit_scenario`	Make targeted edits to a scenario
`delete_scenario`	Remove a scenario
`run_scenario`	Run a scenario, for example after workflow changes

Scenarios are stored alongside the workflow and run instantly without network calls or LLM API usage.

Scenarios in your project

A project workflow’s scenarios live in .reliant/workflows/<name>/scenarios/, one scenario per .yaml file, where <name> is the workflow’s name: as a slug — not its file name. A workflow in blog.yaml that declares name: blog-content-pipeline keeps its scenarios in blog-content-pipeline/scenarios/:

.reliant

workflows

blog.yaml

blog-content-pipeline

scenarios

drafts_from_brief.yaml

rejects_empty_brief.yaml

The app lists these scenarios, and the CLI runs the same ones:

reliant workflow scenario list                 # every project scenario
reliant workflow scenario run                  # run them all; exit 1 on any failure
reliant workflow scenario run .reliant/workflows/blog.yaml --verbose
reliant workflow scenario list --include-builtins  # add the built-in workflows' scenarios

The built-in workflows keep their scenarios in this same layout, and the CLI carries them, so --include-builtins lists or runs them alongside yours from any directory. They are worked examples of every scenario feature on this page. A scenario file’s name: defaults to its file name. A file holding several scenarios is an error — split it into one file per scenario. Scenarios in the retired layouts (scenarios/<file>/ or <file>_scenarios.yaml) are also an error, and the message names the directory each file belongs in. A sub-workflow reference resolves the same way everywhere: ref: project://blog-content-pipeline names the workflow by its name:, in a chat, in a scenario, and in reliant workflow validate. validate follows every reference transitively, so a broken one is reported with the chain that reaches it —

content-weekly.yaml → project://content-next → project://blog: no project workflow is named "blog"; blog.yaml is named "blog-content-pipeline"

— and a reference cycle is an error.

Get started

Use Reliant

Built-in workflows

Machines

Command line

Automations & integrations

Build workflows

Customize

Build & host with Forge

Account & billing

Reference

Help

Testing Workflows

Static Validation

What Gets Validated

Common Validation Errors

When Static Validation Isn’t Enough

Scenario Testing

How It Works

Scenario Structure

Event Types

Event Fields

Targeting Nodes

Inner Nodes (Loops and Sub-Workflows)

How inner nodes execute

Expectations

Partial Testing

Loop Testing Patterns

Single Iteration (Exit Immediately)

Multi-Iteration Loop

Testing Loop Output Routing

Using Scenario Tools

Scenarios in your project

​Static Validation

​What Gets Validated

​Common Validation Errors

​When Static Validation Isn’t Enough

​Scenario Testing

​How It Works

​Scenario Structure

​Event Types

​Event Fields

​Targeting Nodes

​Inner Nodes (Loops and Sub-Workflows)

​How inner nodes execute

​Expectations

​Partial Testing

​Loop Testing Patterns

​Single Iteration (Exit Immediately)

​Multi-Iteration Loop

​Testing Loop Output Routing

​Using Scenario Tools

​Scenarios in your project

Static Validation

What Gets Validated

Common Validation Errors

When Static Validation Isn’t Enough

Scenario Testing

How It Works

Scenario Structure

Event Types

Event Fields

Targeting Nodes

Inner Nodes (Loops and Sub-Workflows)

How inner nodes execute

Expectations

Partial Testing

Loop Testing Patterns

Single Iteration (Exit Immediately)

Multi-Iteration Loop

Testing Loop Output Routing

Using Scenario Tools

Scenarios in your project