anthropic.NewClient() and openai.NewClient() work as they are. LLM providers covers the alternative, a direct provider key of your own.
Before you begin
- A Forge Go API deployed to Reliant hosting, such as the one from Deploy a Go API with managed Postgres.
- Forge v1.6.1 or later in the project (
reliant forge --version). - An organization with hosting billing. The gateway key exists only for environments in an account. An unclaimed deploy without an account gets blank variables.
1
Declare the gateway for the environment
In Repeat it in
deploy/kcl/prod/main.k, give the bundle a hosted LLM, and give the API workload the six gateway variables:deploy/kcl/prod/main.k
staging if you deploy there. forge.llm_env() declares ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY, OPENAI_BASE_URL, OPENAI_API_KEY, RELIANT_LLM_BASE_URL and RELIANT_LLM_API_KEY. Don’t set those names yourself in the same workload: a value you set wins over the platform’s.2
Check the render
terminal
api workload lists the six variables with no value:Output (trimmed)
3
Add the RPC
Declare it in the service’s proto, then run Keep the model name in config, not in code. This project adds an
reliant forge scaffold rpc <service> Summarize to get the stub (see Add a custom RPC). This one is public so you can call it with curl. Leave auth_required at its default in a real app.proto/services/assistant/v1/assistant.proto
llm_model field (env var LLM_MODEL, default claude-haiku-4-5) to its config proto.4
Call the model with the official SDK
Add the SDK with There is no base URL or key in the code.
go get github.com/anthropics/anthropic-sdk-go, then implement the handler:internal/handlers/assistant/rpc_summarize.go
option.WithMaxRetries(0) turns off the SDK’s automatic retries, which would repeat a billed call that an empty wallet refuses again. The OpenAI client (github.com/openai/openai-go) works the same way through OPENAI_BASE_URL and OPENAI_API_KEY.5
Cut a release and deploy it
Commit, then:The first deploy of an environment that declares
terminal
Output (trimmed)
forge.LLM creates its key.6
Call it
Use the A model’s answer, not just an HTTP 200, is what proves the key reached the pod. If the call fails, the API logs the SDK error: see Troubleshooting.
url api: the deploy printed:terminal
Output
Spend, models and billing
Each environment has its own key, shared by every workload in it that readsforge.llm_env(). The gateway enforces the key’s limits:
- Spend cap: $50 per environment per 30 days by default, and the organization’s credit.
- Billing: calls are billed to the organization’s wallet, like every other Reliant model call.
- Models: the gateway’s curated catalog, by its spellings (
claude-haiku-4-5,claude-sonnet-5-5,gemini-3.8-flashand others). Video models are refused to environment keys. - Lifecycle: deleting the environment revokes its key.
What this does not do
- It does not put a key in your frontend. Call the model from the backend only; the browser calls your RPC.
- It does not work on an unclaimed claimable deploy, where the variables stay blank. Write the feature so a missing
ANTHROPIC_API_KEYreturns “unavailable” instead of crashing at startup. - It does not trace the call for you. Neither SDK emits spans;
reliant forge agent skill load llmshows how to wrap a call in apkg/observe/genaispan. - It is not an agent runtime. Multi-step or tool-using work belongs in a Reliant workflow that calls your API.
Troubleshooting
429 insufficient_quota
429 insufficient_quota
The organization is out of Reliant credit. Add credit. Retrying won’t help, which is why the handler turns retries off.
400 or 429 naming a budget
400 or 429 naming a budget
The environment’s spend cap is used up. It resets when the 30-day window does.
401 or 403 naming the model
401 or 403 naming the model
The model isn’t in the key’s catalog. Set
LLM_MODEL to a catalog spelling such as claude-haiku-4-5.503 with Retry-After
503 with Retry-After
The gateway couldn’t load the key. Retry with backoff.
Next steps
LLM providers
Use your own provider key instead of the gateway, and the claimable-deploy limits.
Manage production secrets
Keep any other credential out of git with the managed secret store.
Add a custom RPC
Declare an RPC in proto and scaffold its handler.
Deploy a Go API
Ship the API, its migrations and a managed Postgres as one release.