Model output budgets

Put a cumulative ceiling on the output a team can generate, including its disposable workers and alternative runs. Start with limits for a small task, then adjust the declaration as you learn its needs.

Introduced in v0.6.0. Enforcement is optional and lives at the platform model proxy, with atomic reservations in Postgres shared by all platform replicas.

This is an output-token limit, not a spending limit. Input and cached-input tokens, provider charges and infrastructure costs are not capped. Use a provider-side monetary limit for a dollar ceiling. Calls using separately granted credentials outside the model proxy are outside this boundary.

Declare limits

Add model_budgets inside the organisation resource's spec in Terraform, then plan and apply the organisation stage. These numbers are illustrative, not a recommended spend allowance:

model_budgets = {
  organization_daily_output_tokens = 1000000
  seat_daily_output_tokens         = 200000
  execution_output_tokens          = 20000
  max_request_output_tokens        = 4096
}

Field	Scope
`organization_daily_output_tokens`	All seats in this organisation, resetting at 00:00 UTC
`seat_daily_output_tokens`	Each durable seat identity separately, resetting at 00:00 UTC
`execution_output_tokens`	All calls in one recorded execution, with no midnight reset
`max_request_output_tokens`	The maximum reservation for one request; zero or omitted uses 4096

Zero or omitted cumulative limits are unlimited; at least one of the first three must be positive. All values must be integers from zero through 1,000,000,000,000. The organisation's display time zone does not affect daily accounting. The full schema is in the configuration reference.

Workers and run alternatives share the parent's organisation-daily, seat-daily and original parent-execution counters. Their calls still debit that execution after its parent turn ends, while their worker authority remains valid. Spawning a child does not create fresh quota.

How requests are charged

Before contacting the provider, the proxy reserves the smaller of the request's output ceiling, the configured request cap and the remaining cumulative headroom. If the request omits a ceiling, it starts with the configured cap.
It writes that effective ceiling into the supported provider request. Concurrent calls cannot reserve the same tokens. The response header X-Steadmesh-Output-Token-Limit exposes the ceiling.
After a complete response with valid output usage, it refunds unused reserved tokens. Otherwise the full reservation stays charged.

Missing usage, interrupted streams, upstream errors, client cancellation, failed settlement and platform crashes keep the full reservation. An uncertain request never expires into free quota, and a credential-refresh retry shares the same reservation. Charged usage may therefore be higher than the output you received.

Policy edits and platform restarts retain counters and do not cancel requests already admitted. Removing the block disables enforcement; calls made while disabled are not counted. Re-enabling it uses retained counters for the relevant scope and window.

Inspect and recover from exhaustion

Exhaustion returns HTTP 429 with code budget_exhausted before dispatch to the provider. Daily exhaustion includes Retry-After until the next UTC midnight. You can wait for that reset or raise the declared limit and apply. An execution limit does not reset at midnight: raise it to continue that same execution. New executions have their own execution counters but still share the daily caps.

GET /v1/model-budgets on the platform, authenticated with the seat token and optional execution header, returns policy, charged counters and daily reset time. This is a runtime API, not an orgctl budget command. Charged counters include pending and uncertain reservations. Avoid rapid automatic retries when a scope is exhausted.

Budgeted seat requests require the current lease generation. With an execution limit, they also require a real running execution belonging to that seat; a missing, arbitrary or completed execution ID cannot open quota. Authenticated workers use their recorded parent execution, even after its turn completes, rather than accepting caller-supplied IDs.

Provider compatibility

With budgets enabled, generation is supported only for synchronous, single-completion requests to these endpoints, including streamed SSE responses:

API	Output ceiling
Anthropic `POST /v1/messages`	`max_tokens`
OpenAI `POST /v1/responses`	`max_output_tokens`
OpenAI `POST /v1/chat/completions`	`max_completion_tokens`, or an explicitly supplied legacy `max_tokens`

Unsupported generation paths, multiple completions, background Responses and ambiguous or invalid output-limit fields are rejected before dispatch. Chat streaming requests ask for usage. A legacy Chat backend must receive an explicit max_tokens from its harness; an omitted limit uses max_completion_tokens. A compatible-looking endpoint is not enough: it must honor its output ceiling and report truthful usage.

Model listings, Anthropic token-count requests and connection readiness checks are outside budgeted seat output. Readiness checks can still incur provider charges. The feature does not provide per-work-item budgets, input-token caps or cost accounting. Existing turn timeouts, CPU/memory limits, credential restrictions and network policy remain in force.

For deployment order and database migrations, see Upgrade notes. For model selection and readiness, see Harnesses and models.