> ## Documentation Index
> Fetch the complete documentation index at: https://docs.reliantlabs.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Call an LLM from your deployed app without managing API keys

> Give a hosted Forge API a capped, per-environment model key: declare forge.LLM, add forge.llm_env(), and call the official Anthropic or OpenAI SDK unmodified.

You will add an RPC to a hosted Forge API that asks a model for a one-sentence summary, and deploy it. The app holds no provider key: the platform mints a key for the environment, caps its spend, bills it to your organization, and puts it in the pod where the official SDKs read it.

Reliant's gateway speaks both the OpenAI and Anthropic wire formats, so `anthropic.NewClient()` and `openai.NewClient()` work as they are. [LLM providers](https://docs.reliantlabs.io/forge/llm-provider.md) covers the alternative, a direct provider key of your own.

## Before you begin

* A Forge Go API deployed to Reliant hosting, such as the one from [Deploy a Go API with managed Postgres](https://docs.reliantlabs.io/guides/hosting/deploy-go-api-managed-postgres.md).
* Forge v1.6.1 or later in the project (`reliant forge --version`).
* An organization with [hosting billing](https://docs.reliantlabs.io/hosting/overview.md#billing). The gateway key exists only for environments in an account. An unclaimed [deploy without an account](https://docs.reliantlabs.io/forge/deploy-without-an-account.md) gets blank variables.

**1. Declare the gateway for the environment**

In `deploy/kcl/prod/main.k`, give the bundle a hosted LLM, and give the API workload the six gateway variables:

```kcl deploy/kcl/prod/main.k theme={null}
_api = fw.Workload {
    name = "api"
    # ... as scaffolded
    env = forge.llm_env()
}

_bundle = forge.Bundle {
    # ... as scaffolded
    llm = forge.LLM {runtime = forge.OnHosted {}}
}
```

Repeat it in `staging` if you deploy there. `forge.llm_env()` declares `ANTHROPIC_BASE_URL`, `ANTHROPIC_API_KEY`, `OPENAI_BASE_URL`, `OPENAI_API_KEY`, `RELIANT_LLM_BASE_URL` and `RELIANT_LLM_API_KEY`. Don't set those names yourself in the same workload: a value you set wins over the platform's.

**2. Check the render**

```bash terminal theme={null}
reliant forge env render prod
```

The `api` workload lists the six variables with no value:

```text Output (trimmed) theme={null}
{"name":"RELIANT_LLM_BASE_URL"},{"name":"RELIANT_LLM_API_KEY"},{"name":"OPENAI_BASE_URL"},
{"name":"OPENAI_API_KEY"},{"name":"ANTHROPIC_BASE_URL"},{"name":"ANTHROPIC_API_KEY"}
```

Blank is correct. The platform fills them when it runs the pod. A render that names a different variable, or reads a hosted LLM from a workload that isn't hosted, is refused.

**3. Add the RPC**

Declare it in the service's proto, then run `reliant forge scaffold rpc <service> Summarize` to get the stub (see [Add a custom RPC](https://docs.reliantlabs.io/guides/forge/add-custom-rpc.md)). This one is public so you can call it with curl. Leave `auth_required` at its default in a real app.

```protobuf proto/services/assistant/v1/assistant.proto theme={null}
rpc Summarize(SummarizeRequest) returns (SummarizeResponse) {
  option (forge.v1.method) = {auth_required: false};
}

message SummarizeRequest { string text = 1; }
message SummarizeResponse { string summary = 1; string model = 2; }
```

Keep the model name in config, not in code. This project adds an `llm_model` field (env var `LLM_MODEL`, default `claude-haiku-4-5`) to its config proto.

**4. Call the model with the official SDK**

Add the SDK with `go get github.com/anthropics/anthropic-sdk-go`, then implement the handler:

```go internal/handlers/assistant/rpc_summarize.go theme={null}
func (s *Service) Summarize(
    ctx context.Context,
    req *connect.Request[pb.SummarizeRequest],
) (*connect.Response[pb.SummarizeResponse], error) {
    text := strings.TrimSpace(req.Msg.GetText())
    if text == "" || len(text) > 4000 {
        return nil, svcerr.InvalidArgument("text must be 1-4000 characters")
    }
    ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
    defer cancel()

    // Reads ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY from the environment.
    client := anthropic.NewClient(option.WithMaxRetries(0))
    msg, err := client.Messages.New(ctx, anthropic.MessageNewParams{
        Model:     anthropic.Model(s.deps.Config.LlmModel),
        MaxTokens: 100,
        Messages: []anthropic.MessageParam{
            anthropic.NewUserMessage(anthropic.NewTextBlock("Summarize in one sentence: " + text)),
        },
    })
    if err != nil {
        s.deps.Logger.ErrorContext(ctx, "summarize: model call failed", "error", err)
        return nil, svcerr.Wrap(svcerr.Unavailable("the model is unavailable"))
    }

    var out strings.Builder
    for _, block := range msg.Content {
        if block.Type == "text" {
            out.WriteString(block.Text)
        }
    }
    return connect.NewResponse(&pb.SummarizeResponse{
        Summary: out.String(), Model: string(msg.Model),
    }), nil
}
```

There is no base URL or key in the code. `option.WithMaxRetries(0)` turns off the SDK's automatic retries, which would repeat a billed call that an empty wallet refuses again. The OpenAI client (`github.com/openai/openai-go`) works the same way through `OPENAI_BASE_URL` and `OPENAI_API_KEY`.

**5. Cut a release and deploy it**

Commit, then:

```bash terminal theme={null}
reliant forge env build prod --release v0.3.0
reliant forge env deploy prod v0.3.0 --yes
```

```text Output (trimmed) theme={null}
  url api: https://vivid-puffin.reliantlabs.dev (served through the regional ingress, no CDN)
Rollout of v0.3.0 to prod: SUCCEEDED
  api                      succeeded    1/1 updated
```

The first deploy of an environment that declares `forge.LLM` creates its key.

**6. Call it**

Use the `url api:` the deploy printed:

```bash terminal theme={null}
curl -s -X POST -H 'Content-Type: application/json' \
  -d '{"text":"Reliant hosting runs your API, its migrations and a managed Postgres from one release, and fills DATABASE_URL at deploy time so you never handle a connection string."}' \
  https://vivid-puffin.reliantlabs.dev/services.assistant.v1.AssistantService/Summarize
```

```json Output theme={null}
{"summary":"Reliant hosting automatically manages your API, database migrations, and Postgres infrastructure as a single deployable unit, automatically configuring the database connection without requiring manual setup.","model":"claude-haiku-4-5"}
```

A model's answer, not just an HTTP 200, is what proves the key reached the pod. If the call fails, the API logs the SDK error: see [Troubleshooting](#troubleshooting).

## Spend, models and billing

Each environment has its own key, shared by every workload in it that reads `forge.llm_env()`. The gateway enforces the key's limits:

* **Spend cap:** \$50 per environment per 30 days by default, and the organization's credit.
* **Billing:** calls are billed to the organization's wallet, like every other Reliant model call.
* **Models:** the gateway's curated catalog, by its spellings (`claude-haiku-4-5`, `claude-sonnet-5-5`, `gemini-3.8-flash` and others). Video models are refused to environment keys.
* **Lifecycle:** deleting the environment revokes its key.

## What this does not do

* It does not put a key in your frontend. Call the model from the backend only; the browser calls your RPC.
* It does not work on an unclaimed claimable deploy, where the variables stay blank. Write the feature so a missing `ANTHROPIC_API_KEY` returns "unavailable" instead of crashing at startup.
* It does not trace the call for you. Neither SDK emits spans; `reliant forge agent skill load llm` shows how to wrap a call in a `pkg/observe/genai` span.
* It is not an agent runtime. Multi-step or tool-using work belongs in a [Reliant workflow](https://docs.reliantlabs.io/guides/workflows/first-multi-agent-workflow.md) that calls your API.

## Troubleshooting

> [accordion] 429 insufficient\_quota
>
> The organization is out of Reliant credit. Add credit. Retrying won't help, which is why the handler turns retries off.

> [accordion] 400 or 429 naming a budget
>
> The environment's spend cap is used up. It resets when the 30-day window does.

> [accordion] 401 or 403 naming the model
>
> The model isn't in the key's catalog. Set `LLM_MODEL` to a catalog spelling such as `claude-haiku-4-5`.

> [accordion] 401 unauthorized
>
> The key is missing or revoked. Check that the workload uses `forge.llm_env()`, that nothing else sets those names, and that the bundle declares `llm`.

> [accordion] 503 with Retry-After
>
> The gateway couldn't load the key. Retry with backoff.

## Next steps

> [card] [LLM providers](https://docs.reliantlabs.io/forge/llm-provider.md)
>
> Use your own provider key instead of the gateway, and the claimable-deploy limits.

> [card] [Manage production secrets](https://docs.reliantlabs.io/guides/hosting/manage-production-secrets.md)
>
> Keep any other credential out of git with the managed secret store.

> [card] [Add a custom RPC](https://docs.reliantlabs.io/guides/forge/add-custom-rpc.md)
>
> Declare an RPC in proto and scaffold its handler.

> [card] [Deploy a Go API](https://docs.reliantlabs.io/guides/hosting/deploy-go-api-managed-postgres.md)
>
> Ship the API, its migrations and a managed Postgres as one release.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.