What is collected, and how often
The profiler is part of forge’s runtime library, started when a service sets up its telemetry. It runs on its own goroutine, never blocks a request, and backs off if the collector is unreachable.
These are sparse on purpose, because every stored sample repeats its whole stack and profiles are the costliest signal per useful byte. A long-lived process has roughly a minute of CPU capture an hour. That is enough to find a sustained hotspot or a steady heap leak, and it can miss a short incident or a single slow request entirely. To look harder for a while, raise the cadence on one workload through its
env (FORGE_PROFILES_CPU_INTERVAL, at least 30s) and put it back afterwards; Where the telemetry goes lists the variables.
The allocation counters are off because Go reports them cumulatively since process start. Every snapshot repeats every allocation site the process ever had, so the cost grows with uptime. Turn them on for one workload when you are hunting allocations, and turn them off when you are done. Agents can load observability/profiles for the workload profiling block that does it.
Is it on?
It depends on where the environment’s telemetry goes. The environment’sobservability block decides, like every other signal, and signals.profiles is the switch.
So the five-minute CPU and two-hour heap cadence is what a process does once profiling is on, and a hosted environment does not collect profiles until you opt in. To turn it on for an environment, declare the signal in its
main.k and deploy:
OTEL_PROFILES_EXPORTER=none or FORGE_PROFILES=none in its env.
Profiles are billed as telemetry once they are collected, and a hosted environment that is over its observability cap stops storing them first. Retention is 24 hours for raw samples and 30 days for the per-minute aggregates behind range flamegraphs. Observability billing has the included amounts and the cap.
Reading a profile
Local and hosted profiles are read differently. Locally, the CLI exports them from the stack on your machine. On hosting, you read them in the Reliant app.Locally, with forge obs profile
forge env up keeps profiles in the same local store as your other telemetry, but the HyperDX UI cannot draw flamegraphs. forge obs profile exports them in a form other tools can open. It needs the local stack running.
--format is pprof (the default), speedscope or folded. Open the speedscope file at speedscope.app, or pipe the folded stacks to flamegraph.pl for an SVG. The default --type is cpu; inuse_space and inuse_objects show the live heap, and the allocation, goroutine, mutex and block types appear only for a workload that turned them on. --since defaults to one hour, and --from and --to take RFC 3339 times. --sql prints the query instead of running it, so you can run it yourself.
CPU time is summed over the range. The live heap and goroutines are different: they are snapshots, so the export uses each instance’s newest snapshot in the range, and looks back up to 24 hours when the range holds none. A one-hour window therefore still shows the heap even though snapshots are two hours apart.
To see where one slow request spent its CPU, narrow to a trace, which uses the raw samples from the last 24 hours:
On hosting, in the Reliant app
Open Apps, pick your project and environment, and choose Profiles in the Observability section. It draws a flamegraph of a service’s CPU, memory or goroutines over the time range you pick, with a table of the heaviest functions beside it. Opening it from a trace narrows it to that trace’s samples. Where the telemetry goes describes the rest of the Apps navigation. The page is a view over the control plane’sGetFlamegraph call, which returns a service’s aggregated profile as speedscope JSON. Reading it requires sign-in. forge obs profile reads only the local stack: its --env flag keeps the samples whose environment name matches, and does not export a hosted environment’s profiles.
Related topics
- Where the telemetry goes: the local store, the hosted store, retention, profile volume and opting out.
- Querying with forge obs: the other
forge obsverbs, includingcheck,statusandtrace. - Hosted dashboards, alerts and issues: the rest of what Apps shows for an environment.
- Observability billing: included amounts, the cap and what stops when it is reached.