forge obs endpoints can rank all of them by p95 latency, and the slowest traces show where the time went. This guide finds an N+1 query this way, on a local stack and in production.
The same verbs read dev (the stack reliant forge env up dev runs on your machine) and a hosted environment such as prod: the env’s declaration picks the backend. Querying with forge obs covers every verb.
Before you begin
- A Forge API with some traffic: running locally with
reliant forge env up dev(Build a Go and React app), or deployed (Deploy a Go API with managed Postgres). - Forge v1.6.1 or later in the project (
reliant forge --version). - For a hosted environment,
obs:readon the organization. Under Reliant your sign-in already carries it.
GetCatalogReport lists 200 products and then loads each one again by ID, and FingerprintCatalog hashes its input a million times.
1
Rank endpoints by p95
terminal
Output
--sort takes requests, p95, errors or rate, and --top sets how many rows (default 10). Use dev instead of prod for your local stack.2
List the slowest traces for one RPC
--route matches the root span name, and --slow sorts slowest first:terminal
Output (trimmed)
--min-duration 500ms keeps only traces at least that long.3
Read one trace
terminal
Output (trimmed)
orm.GetProduct spans into one line with their spread: one query per product after the list, an N+1. The * marks the critical path.A slow RPC with no child spans looks different. Here is FingerprintCatalog on the local stack:Output
4
See the query behind any verb
--explain (alias --print-sql) prints the compiled SQL and its parameters instead of running it:terminal
Output (trimmed)
obs endpoints and obs traces take --explain too. Copy a statement into reliant forge obs query prod --sql "…" when the structured verbs cannot express what you need, for example grouping by a span attribute.5
Fix it and verify
Here the fix was to total the prices from the list instead of loading each product again. After the change, the same two reads on the local stack:
terminal
Output
GetCatalogReport’s p95 fell from 29ms to 1.1ms locally, and each trace has 3 spans instead of 403. In production, deploy the fix as a new release and run reliant forge obs check prod --wait --json, which compares p95 per endpoint before and after the promotion: How release verdicts work.Numbers differ between environments. Locally the API had the machine’s CPU; the hosted
api workload requests a quarter of a core, so the same hashing loop took 80ms on the laptop and 1.4s in production. Compare an RPC against itself in one environment, not across them.What this does not do
obs endpointsranks server spans that your services exported. An app Forge did not build has no rows until it sends OpenTelemetry traces: Send OpenTelemetry from any app.- It reads; it does not alert. Latency alerts and dashboards are generated from the code: Hosted dashboards, alerts and issues.
- A slow release is not rolled back for you. You recover by deploying a fixed or earlier release.
Troubleshooting
exit 2 from obs endpoints
exit 2 from obs endpoints
Forge could not look: the local stack is down (start it with
reliant forge env up dev), the credential lacks obs:read, or a read was rate limited. It does not mean “no slow endpoints”. See the exit codes./bin/sh: 1: forge: not found
/bin/sh: 1: forge: not found
env up dev has only reliant forge to call. The fix is in Build a Go and React app.Next steps
Find CPU hotspots
When a trace says “uninstrumented”, find the function with a CPU profile.
Debug production with an AI agent
Let an agent run these reads and follow a failing request to the line.
Querying with forge obs
Every verb, the JSON contract and the exit codes.
How release verdicts work
How
obs check judges p95 per endpoint after a deploy.