Skip to main content
You will add an RPC to a hosted Forge API that asks a model for a one-sentence summary, and deploy it. The app holds no provider key: the platform mints a key for the environment, caps its spend, bills it to your organization, and puts it in the pod where the official SDKs read it. Reliant’s gateway speaks both the OpenAI and Anthropic wire formats, so anthropic.NewClient() and openai.NewClient() work as they are. LLM providers covers the alternative, a direct provider key of your own.

Before you begin

1

Declare the gateway for the environment

In deploy/kcl/prod/main.k, give the bundle a hosted LLM, and give the API workload the six gateway variables:
deploy/kcl/prod/main.k
Repeat it in staging if you deploy there. forge.llm_env() declares ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY, OPENAI_BASE_URL, OPENAI_API_KEY, RELIANT_LLM_BASE_URL and RELIANT_LLM_API_KEY. Don’t set those names yourself in the same workload: a value you set wins over the platform’s.
2

Check the render

terminal
The api workload lists the six variables with no value:
Output (trimmed)
Blank is correct. The platform fills them when it runs the pod. A render that names a different variable, or reads a hosted LLM from a workload that isn’t hosted, is refused.
3

Add the RPC

Declare it in the service’s proto, then run reliant forge scaffold rpc <service> Summarize to get the stub (see Add a custom RPC). This one is public so you can call it with curl. Leave auth_required at its default in a real app.
proto/services/assistant/v1/assistant.proto
Keep the model name in config, not in code. This project adds an llm_model field (env var LLM_MODEL, default claude-haiku-4-5) to its config proto.
4

Call the model with the official SDK

Add the SDK with go get github.com/anthropics/anthropic-sdk-go, then implement the handler:
internal/handlers/assistant/rpc_summarize.go
There is no base URL or key in the code. option.WithMaxRetries(0) turns off the SDK’s automatic retries, which would repeat a billed call that an empty wallet refuses again. The OpenAI client (github.com/openai/openai-go) works the same way through OPENAI_BASE_URL and OPENAI_API_KEY.
5

Cut a release and deploy it

Commit, then:
terminal
Output (trimmed)
The first deploy of an environment that declares forge.LLM creates its key.
6

Call it

Use the url api: the deploy printed:
terminal
Output
A model’s answer, not just an HTTP 200, is what proves the key reached the pod. If the call fails, the API logs the SDK error: see Troubleshooting.

Spend, models and billing

Each environment has its own key, shared by every workload in it that reads forge.llm_env(). The gateway enforces the key’s limits:
  • Spend cap: $50 per environment per 30 days by default, and the organization’s credit.
  • Billing: calls are billed to the organization’s wallet, like every other Reliant model call.
  • Models: the gateway’s curated catalog, by its spellings (claude-haiku-4-5, claude-sonnet-5-5, gemini-3.8-flash and others). Video models are refused to environment keys.
  • Lifecycle: deleting the environment revokes its key.

What this does not do

  • It does not put a key in your frontend. Call the model from the backend only; the browser calls your RPC.
  • It does not work on an unclaimed claimable deploy, where the variables stay blank. Write the feature so a missing ANTHROPIC_API_KEY returns “unavailable” instead of crashing at startup.
  • It does not trace the call for you. Neither SDK emits spans; reliant forge agent skill load llm shows how to wrap a call in a pkg/observe/genai span.
  • It is not an agent runtime. Multi-step or tool-using work belongs in a Reliant workflow that calls your API.

Troubleshooting

The organization is out of Reliant credit. Add credit. Retrying won’t help, which is why the handler turns retries off.
The environment’s spend cap is used up. It resets when the 30-day window does.
The model isn’t in the key’s catalog. Set LLM_MODEL to a catalog spelling such as claude-haiku-4-5.
The key is missing or revoked. Check that the workload uses forge.llm_env(), that nothing else sets those names, and that the bundle declares llm.
The gateway couldn’t load the key. Retry with backoff.

Next steps

LLM providers

Use your own provider key instead of the gateway, and the claimable-deploy limits.

Manage production secrets

Keep any other credential out of git with the managed secret store.

Add a custom RPC

Declare an RPC in proto and scaffold its handler.

Deploy a Go API

Ship the API, its migrations and a managed Postgres as one release.