LLM Integration Guide

This guide covers how Reliant interacts with LLMs, common caveats, and provider-specific considerations.

How LLM Calls Work

When Reliant calls an LLM, it sends a structured request:

┌─────────────────────────────────────┐
│ LLM Request                         │
├─────────────────────────────────────┤
│ System Prompt (from workflow/preset)│
│ ─────────────────────────────────── │
│ Message History (thread context)    │
│   - User messages                   │
│   - Assistant responses             │
│   - Tool calls & results            │
│ ─────────────────────────────────── │
│ Available Tools (definitions)       │
│ ─────────────────────────────────── │
│ Parameters (temperature, etc.)      │
└─────────────────────────────────────┘

The LLM returns:

Text response (assistant message)
Tool calls (requests to use tools)
Stop reason (why the turn ended: tool_use, incomplete, truncated, interrupted, refused, error, done — see CEL Expressions)

Tool Call Requirements

A model can only use the tools it was offered on that turn. A call_llm node says what those are in tools_config:

- id: call_llm
  type: call_llm
  args:
    model: {tags: [flagship]}
    tools_config:
      preloaded_tools: ["view", "edit", "tag:shell"]   # handed to the model up front
      loadable_tools: ["*"]                            # what load_tool may add later

Tool Matching Rules

Tool calls reference tools by name
The name must exactly match a tool offered on that turn
Tool inputs must match the tool’s schema
A call to a tool that was not offered is refused, never executed: the model gets an error result saying so (“Tool ‘write’ was not offered on this turn”) and can carry on

Best Practice

preloaded_tools is a starting bundle, not a boundary: list what the agent needs in hand, and let loadable_tools decide what it may reach for with load_tool. Leaving loadable_tools out means everything is loadable; an empty list means exactly what was preloaded and nothing more.

# The standard coding set, with anything else loadable on demand
preloaded_tools: ["tag:coding:default"]

# Exactly these tools, nothing more
preloaded_tools: ["view", "tag:shell"]
loadable_tools: []

Entries can be tool names, tags (tag:coding:default, tag:shell, tag:web), globs (mcp__github__*) and exclusions (!tag:shell). The Tools reference lists every tool and its tags.

System Messages

System prompts set the LLM’s behavior and context. They’re sent first in every request.

Where System Prompts Come From

Node-level system_prompt on a call_llm node
The workflow’s system_prompt input, which a workflow passes to its call_llm node. The built-in agent’s is empty by default, which means Reliant’s own default prompt
A preset that sets system_prompt, and a node’s args, which override the preset’s value

Reliant adds context of its own to every agent call: the project or worktree it works in, its machine, and the project’s memory files.

System Prompt Best Practices

system_prompt: |
  # Role
  You are a code reviewer focusing on security.
  
  # Rules
  - Always check for injection vulnerabilities
  - Flag hardcoded credentials
  - Suggest secure alternatives
  
  # Output Format
  Provide findings in markdown with severity levels.

Tips:

Be specific about the task
Define output format expectations
Include examples if needed
Keep it focused (long prompts waste tokens)

Context and Tokens

Context Window

Each model has a maximum context window (total tokens for input + output). The Models reference lists every model Reliant knows with its context window, generated from the same registry Reliant selects models from.

Token Counting

Reliant tracks token usage per thread. When context grows large:

Compaction summarizes older messages once the thread passes the model’s compaction threshold (compaction_threshold on the Call LLM node’s output). The built-in agent does this on its own, with the loop’s own model
Filtering reduces tool result verbosity, when a workflow adds a filtering step (see Multi-Agent Patterns)

Caching

Providers that support prompt caching reuse a repeated prefix (the system prompt, tools and earlier messages) at a lower price. Reliant uses it automatically wherever the provider offers it; the Models reference marks those models with 💾.

Provider Differences

Reliant drives every provider through the same request and output shape, so a workflow does not change when its model does: tool calls arrive as tool_calls, and every provider’s way of ending a turn is mapped onto the same stop_reason values. What does differ between models is listed per model in the Models reference: context window, attachments, prompt caching, and whether and how hard it can think (thinking_level is clamped to the levels a model supports). Select models by tag rather than by ID, as the built-in workflows and presets do, so a workflow keeps working with whichever providers a user has connected.

Common Pitfalls

1. Tool Call Without Matching Tool

# Problem: the model needs a tool it was not offered, and loading is closed off
tools_config:
  preloaded_tools: ["view"]
  loadable_tools: []

# Solution: preload what it needs, or let it load the rest on demand
tools_config:
  preloaded_tools: ["tag:coding:default"]
  loadable_tools: ["*"]

2. Context Overflow

# Problem: Large tool results fill context
- id: search
  type: run
  command: grep -r "pattern" .  # Could return megabytes

# Solution: Limit results or filter
- id: search
  type: run
  command: grep -r "pattern" . | head -100

3. Infinite Loops

# Problem: No termination condition
while: "true"

# Solution: Always include iteration limit
while: "iter.iteration < inputs.max_turns && ..."

4. Empty Tool Calls Array

# Problem: Assuming tool_calls is never null
condition: size(nodes.llm.tool_calls) > 0

# Solution: Check for null first
condition: nodes.llm.tool_calls != null && size(nodes.llm.tool_calls) > 0

5. Deciding Whether the Model Is Done

# Problem: zero tool calls does NOT mean the model is done.
# A turn with no tool calls can also be:
# - truncated     (hit max_tokens mid-thought)
# - interrupted   (the stream was cut short)
# - incomplete    (the provider paused the turn and expects it back —
#                  e.g. "I'll check the config next." with no tool call yet)

# Solution: loop on stop_reason, which call_llm derives from all of these.
while: "outputs.stop_reason in ['tool_use', 'incomplete', 'truncated', 'interrupted']"

# Routing to execute_tools still keys on the calls themselves:
condition: size(nodes.llm.tool_calls) > 0

Response Tools

A response tool asks the model for structured output: a tool only the model can call, whose input is the answer you described with a JSON Schema.

response_tool:
  name: decision
  description: Report your decision
  schema:
    type: object
    required: [choice, value]
    properties:
      choice: {type: string, enum: [approve, reject]}
      value: {type: string, description: Why}

When the model calls it, the Call LLM node parses the call into response_data:

{
  "choice": "approve",
  "value": "The code follows all conventions and has good test coverage."
}

The tool is offered, not forced, so a model can still answer in text. Guard reads with has(nodes.<id>.response_data.choice), or use builtin://structured-agent, which keeps reminding the model until it answers through the tool. Use for:

Routing decisions
Classification tasks
Structured data extraction

See Workflow Examples for patterns.

Debugging LLM Issues

Look at the Run

Open the run in Workflows → Runs. Its page shows the transcript and how the run was launched, and Diagram shows the workflow’s steps as they ran. The context gauge in the chat header shows how much of the model’s window the thread is using. From a terminal, reliant workflow status <execution-id> prints a snapshot of a run and reliant workflow follow <execution-id> streams its events as they happen.

Common Fixes

Symptom	Likely Cause	Fix
”Tool ’…’ was not offered on this turn”	Tool not preloaded, and not loadable	Add it to `preloaded_tools`, or allow it in `loadable_tools`
Truncated output	Hit `max_tokens`	Raise `max_tokens` on the Call LLM node; loop on `stop_reason` so a truncated turn continues
Repeated responses	No stop condition	Add iteration limits to loops
Slow responses	Large context	Enable compaction or filter results
Wrong tool called	Ambiguous prompt	Clarify in system prompt

See also: CEL Expressions

Get started

Use Reliant

Built-in workflows

Machines

Command line

Automations & integrations

Build workflows

Customize

Build & host with Forge

Account & billing

Reference

Help

LLM Integration Guide

How LLM Calls Work

Tool Call Requirements

Tool Matching Rules

Best Practice

System Messages

Where System Prompts Come From

System Prompt Best Practices

Context and Tokens

Context Window

Token Counting

Caching

Provider Differences

Common Pitfalls

1. Tool Call Without Matching Tool

2. Context Overflow

3. Infinite Loops

4. Empty Tool Calls Array

5. Deciding Whether the Model Is Done

Response Tools

Debugging LLM Issues

Look at the Run

Common Fixes

​How LLM Calls Work

​Tool Call Requirements

​Tool Matching Rules

​Best Practice

​System Messages

​Where System Prompts Come From

​System Prompt Best Practices

​Context and Tokens

​Context Window

​Token Counting

​Caching

​Provider Differences

​Common Pitfalls

​1. Tool Call Without Matching Tool

​2. Context Overflow

​3. Infinite Loops

​4. Empty Tool Calls Array

​5. Deciding Whether the Model Is Done

​Response Tools

​Debugging LLM Issues

​Look at the Run

​Common Fixes

How LLM Calls Work

Tool Call Requirements

Tool Matching Rules

Best Practice

System Messages

Where System Prompts Come From

System Prompt Best Practices

Context and Tokens

Context Window

Token Counting

Caching

Provider Differences

Common Pitfalls

1. Tool Call Without Matching Tool

2. Context Overflow

3. Infinite Loops

4. Empty Tool Calls Array

5. Deciding Whether the Model Is Done

Response Tools

Debugging LLM Issues

Look at the Run

Common Fixes