> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reliantlabs.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Run a local LLM coding agent with Ollama, LM Studio or vLLM

> Serve a model with Ollama, LM Studio, llama.cpp or vLLM on your own machine, then run a workflow where a local model drafts and a cloud model reviews.

By the end of this guide a model running on your own hardware answers one step of a workflow, and a cloud model answers the next. Use it to keep cheap, high-volume steps on a local GPU while the hard steps go to a frontier model.

Reliant's servers make every model call, so they reach a local model through a machine you connected: its daemon finds the model server on that machine and relays requests to it ([AI providers](https://docs.reliantlabs.io/settings/api-keys.md#local-models)).

## Before you begin

* A machine that can reach the model server, usually your own computer, running the desktop app or `reliant daemon start` ([connect your computer](https://docs.reliantlabs.io/guides/machines/connect-computer-as-machine.md)). A cloud machine can't see a server on your laptop.
* For the cloud step, any connected provider: [a subscription](https://docs.reliantlabs.io/guides/machines/use-claude-or-chatgpt-subscription.md), an API key or Reliant AI.
* The `reliant` CLI v1.21 or later, signed in ([Sign in and tokens](https://docs.reliantlabs.io/cli/sign-in.md)).

**1. Serve a model on your machine**

Start Ollama with a context window big enough for agents, and pull a model. This guide uses a 0.5B model because it downloads in seconds; pick a larger, tool-capable model for real work.

```bash terminal theme={null}
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
ollama pull qwen2.5:0.5b
curl -s http://localhost:11434/v1/models
```

```text terminal theme={null}
{"object":"list","data":[{"id":"qwen2.5:0.5b","object":"model","created":1795168794,"owned_by":"library"}]}
```

LM Studio, llama.cpp and vLLM work the same way: start their OpenAI-compatible server and load a model.

**2. Let your machine find it**

The daemon checks `localhost` on ports 11434 (Ollama), 1234 (LM Studio), 8080 (llama.cpp) and 8000 (vLLM) when it connects and every minute after, and publishes the chat models it finds. Open **Settings → AI providers → Local & custom** to see them, with each model's context size and whether it supports tools.

From a terminal, `reliant models list --local` prints the same models with their ids and the machine serving each, and `reliant models endpoint list` shows each server a machine found, or why it couldn't reach one.

A server on another port or host isn't detected. Add it there: **Add endpoint** for an address your machine reaches (or `reliant models endpoint add http://gpu-box.lan:8000/v1 --machine <machine>`), or **Add custom endpoint** for a server on the public internet (**Reliant cloud**) or on your LAN or VPN (**Through one of my machines**).

> [!NOTE]
>
> Settings flags a model whose context is under 64K, and calls one under 16K too small for agent work. For Ollama, raise it with `OLLAMA_CONTEXT_LENGTH` or the app's **Context length** setting, then click **Test connection**.

**3. Name the local model in a workflow step**

A detected model's id is its name on the server plus `@local`. Here the first step runs on Ollama and the second on the `fast` tier of whatever cloud provider you have:

```yaml .reliant/workflows/local-draft-cloud-review.yaml theme={null}
name: local-draft-cloud-review
description: A local model drafts a commit message, a cloud model reviews it
entry: [draft]

nodes:
  # Step 1 runs on the Ollama server your machine can see.
  - id: draft
    type: workflow
    ref: builtin://agent
    args:
      model:
        id: "qwen2.5:0.5b@local"
      tools: []            # no tools: a small local model only writes text here
      spawn_presets: []
    thread:
      mode: new
      inject:
        role: user
        content: "Write a one-line git commit message for this change: added a README that explains how to run the tests. Reply with the message only."

  # Step 2 runs on the fastest cloud model you have a provider for.
  - id: review
    type: workflow
    ref: builtin://agent
    args:
      model:
        tags: [fast]
      tools: []
      spawn_presets: []
    thread:
      mode: new
      inject:
        role: user
        content: "Is this a good one-line commit message? Answer yes or no, then one sentence why.\n\n{{nodes.draft.response_text}}"

  # Writes both answers to a file in the project, so you can check them.
  - id: record
    type: run
    command: |
      cat > model-check.txt <<'EOF'
      draft:  {{nodes.draft.response_text}}
      review: {{nodes.review.response_text}}
      EOF
    timeout: 1m

edges:
  - from: draft
    default: review
  - from: review
    default: record
```

```bash terminal theme={null}
reliant workflow validate .reliant/workflows/local-draft-cloud-review.yaml
```

Validation checks the shape, not whether a machine serves the model; that is decided when the step runs.

**4. Run it and check which model answered**

Run it on the machine that serves the model. Add `--daemon <machine>` if that isn't your default machine:

```bash terminal theme={null}
reliant workflow run local-draft-cloud-review --project-path "$PWD" --follow
cat model-check.txt
```

```text terminal theme={null}
Workflow "local-draft-cloud-review" triggered
...
{"event":"workflow_completed",...,"workflow_name":"local-draft-cloud-review","old_state":"running","new_state":"completed",...}
draft:  * Added a README file with instructions for running tests in the `local-models/guides-sandbox-localmodel` directory.
review: No. The message describes what was added rather than why, and "good one-line" commit messages follow conventional commits format (e.g., "docs: add test instructions for local model sandbox") to categorize the change type.
```

Ollama's own log shows the draft step reached it:

```text terminal theme={null}
[GIN] 2026/11/20 - 10:32:47 | 200 |    10.8s |       127.0.0.1 | POST     "/v1/chat/completions"
```

The 0.5B model's draft is rough, which is what the cloud review step is for.

## What this does not do

* **It doesn't keep your prompts off Reliant's servers.** Only inference runs on your hardware. The prompt and the reply pass through Reliant and are stored in your chat ([Privacy & Security](https://docs.reliantlabs.io/help/privacy.md#local-models)).
* **It doesn't work while that machine is away.** The model is reached through the machine that found it. When that machine is offline or the server stops, the step fails until it is back.
* **It doesn't make a small model an agent.** A step that edits files needs a model that supports tools and a 64K or larger context. The steps above give the agents no tools.

## Troubleshooting

> [accordion] Local model qwen2.5:0.5b is unavailable: none of your machines serve it (is the model pulled and the server running?)
>
> None of your connected machines published that model. Start the server, check the name with `curl http://localhost:11434/v1/models`, and wait up to a minute for the next probe. The run retries the step a few times and then pauses. Once the model shows up, continue it with `reliant workflow resume <execution-id>`.

> [accordion] Local model … on … is unavailable: that machine is offline
>
> The only machine that serves the model isn't connected. Start the desktop app or `reliant daemon start` on it ([fix an offline machine](https://docs.reliantlabs.io/guides/machines/troubleshoot-offline-machine.md)).

> [accordion] no machine has synced this project's workflows yet
>
> `--project-path` has just created the project, and its machine hasn't sent the workflows yet. Wait a minute and run the command again.

## Next steps

> [card] [Use your Claude or ChatGPT subscription](https://docs.reliantlabs.io/guides/machines/use-claude-or-chatgpt-subscription.md)
>
> Connect the cloud side of the workflow with a plan you already pay for.

> [card] [Use a different model for each step](https://docs.reliantlabs.io/guides/workflows/choose-model-per-step.md)
>
> Tags, explicit ids and thinking levels per node.

> [card] [Write your first multi-agent workflow](https://docs.reliantlabs.io/guides/workflows/first-multi-agent-workflow.md)
>
> A planner, an implementer and a reviewer in one file.

> [card] [Model configuration](https://docs.reliantlabs.io/settings/model-config.md)
>
> Custom models, tier order and `~/.reliant/config.yaml`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.