Cost controls

AI spend has three shapes of failure: one run runs away, a whole workflow runs away, or a burst saturates your provider quota. Duraton gives each its own control, declared on defineWorkflow next to retry, plus two per-call options that cut spend and absorb provider failures.

defineWorkflow({
  name: "summarize",
  cap: { maxCost: 0.25 },                                  // ceiling on ONE run's spend -> fail
  budget: { maxCost: 5, windowMs: 3_600_000 },             // ceiling on the workflow's spend -> pause
  tokenThrottle: { tokens: 100_000, perMs: 60_000 },       // token rate -> delay the start
  handler,
});

The reference tables for the three workflow fields live in flow control; the per-call options are in step.ai.generate. This page is how to choose between them.

cap - bound one run

cap is a hard ceiling on a single run's AI spend. The run halts before the step.ai call that would cross a set axis - that call never runs - and fails with a BudgetError. Everything committed before the halt stays committed.

defineWorkflow({
  name: "research.agent",
  cap: { maxCost: 0.25, maxTokens: 40_000 },
  handler: async (ctx) => {
    await ctx.step.ai.loop("agent", {
      prompt: ctx.event.data.brief,
      maxIterations: 20,
      tools: { search: { handler: (q) => search(q) } },
      turn: (c, i) => callModel(c.prompt, c.history, i),
    });
  },
});

from duraton import CapConfig, LoopHandlerTool, LoopOptions, define_workflow
from duraton.context import StepContext


async def research(ctx: StepContext) -> object:
    await ctx.step.ai.loop("agent", LoopOptions(
        prompt=ctx.event.data["brief"],
        max_iterations=20,
        tools={"search": LoopHandlerTool(handler=lambda q: search(q))},
        turn=lambda c, i: call_model(c.prompt, c.history, i),
    ))
    return None


research_agent = define_workflow(
    "research.agent",
    research,
    cap=CapConfig(max_cost=0.25, max_tokens=40_000),
)

var (
	capMaxCost   = 0.25
	capMaxTokens = int64(40_000)
)

var researchAgent = duraton.DefineWorkflow(duraton.WorkflowDefinition{
	Name: "research.agent",
	Cap:  &duraton.CapConfig{MaxCost: &capMaxCost, MaxTokens: &capMaxTokens},
	Handler: func(c *duraton.Context) (any, error) {
		_, err := duraton.Loop[any](c, "agent", duraton.LoopOptions{
			Prompt:        brief,
			MaxIterations: 20,
			Tools: map[string]duraton.LoopTool{
				"search": {Handler: func(in any) (any, error) { return search(in) }},
			},
			Turn: func(lc duraton.LoopContext, i int) (duraton.LoopTurn, error) {
				return callModel(lc.Prompt, lc.History, i)
			},
		})
		return nil, err
	},
})

At least one axis is set. The ceiling is crossed by at most the one call that reaches it: a call's cost is unknown until it returns, so the call that pushes spend to the limit completes and the next one halts.

What you see. A capped run is an ordinary failed run in the console, carrying BudgetError as its terminal error - there is no separate "capped" state. In an agent loop, the halted turn is the loop's last (failed) iteration. Raise the cap and replay the run and it starts fresh with spend back at zero.

budget - bound a workflow over a window

Where cap bounds one run, budget bounds the workflow's aggregate spend across all its runs over a rolling window. When a set axis is crossed, Duraton pauses the workflow's in-flight runs at their checkpoint - holding no runner, keeping their progress - and resumes them on its own when the window rolls forward or you raise the budget.

defineWorkflow({
  name: "support.autoreply",
  budget: { maxCost: 5, windowMs: 3_600_000, warnAtPct: 80 }, // $5/hour, amber at $4
  handler,
});

# Python first-classes retry/concurrency/cap; every other knob rides extra= with wire keys.
support_autoreply = define_workflow(
    "support.autoreply",
    handler,
    extra={"budget": {"maxCost": 5, "windowMs": 3_600_000, "warnAtPct": 80}},
)

var budgetMaxCost = 5.0

var supportAutoreply = duraton.DefineWorkflow(duraton.WorkflowDefinition{
	Name:    "support.autoreply",
	Budget:  &duraton.BudgetConfig{MaxCost: &budgetMaxCost, WindowMs: 3_600_000, WarnAtPct: 80},
	Handler: handler,
})

At least one axis is set. Duraton checks the budget on a background loop, so overshoot is bounded by one check interval (plus any per-run cap).

What you see. A budget-paused run is an ordinary paused run whose pausedBy.reason is "budget" - the console, GET /runs/{id}, and the get_run MCP tool all show it, so it is distinguishable from a manual pause. It resumes automatically; a manual resume is only an override, and a manually paused run is never auto-resumed.

tokenThrottle - bound the rate

A tokenThrottle protects your own provider quota: at most tokens spent per perMs across the runs sharing a key. Duraton debits each AI step's actual token usage after the step commits, and when the bucket is drained it delays new run starts for that key - the run waits in the queue holding no runner, then runs normally.

defineWorkflow({
  name: "enrich.contact",
  tokenThrottle: { tokens: 100_000, perMs: 60_000, key: "customerId" }, // 100k tokens/min per customer
  handler,
});

enrich_contact = define_workflow(
    "enrich.contact",
    handler,
    extra={"tokenThrottle": {"tokens": 100_000, "perMs": 60_000, "key": "customerId"}},
)

var enrichContact = duraton.DefineWorkflow(duraton.WorkflowDefinition{
	Name:          "enrich.contact",
	TokenThrottle: &duraton.TokenThrottleConfig{Tokens: 100_000, PerMs: 60_000, Key: "customerId"},
	Handler:       handler,
})

Because a step's tokens are known only after it runs, the throttle gates a run's start on the key's recent usage: a key that has recently spent heavily has its next runs spread out, a fresh key starts immediately. It shapes the rate of starts, not any single run - pair it with a cap to also bound one run.

What you see. A throttled run sits in queued with a future start time, then runs normally. There is no new state to handle.

cache - don't pay twice for the same call

The inference cache is the one control that reduces spend rather than bounding it. On a hit the provider is never called, so the step commits with zero spend and counts nothing against cap, budget, or tokenThrottle. Where step memoization makes a replay free, the cache makes an identical call in a different run free too.

const answer = await ctx.step.ai.generate("answer", {
  model: "claude-opus-4-8",
  prompt: `Answer from this policy doc:\n${doc}\n\nQ: ${question}`,
  temperature: 0, // required - caching engages only for a deterministic call
  cache: { ttlMs: 3_600_000 },
});

temp := 0.0 // required - caching engages only for a deterministic call
answer, err := duraton.Generate(c, "answer", duraton.GenerateOptions{
	Model:       "claude-opus-4-8",
	Prompt:      fmt.Sprintf("Answer from this policy doc:\n%s\n\nQ: %s", doc, question),
	Temperature: &temp,
	Cache:       &duraton.CacheOptions{TTL: time.Hour},
})

The key is an exact match over the seed, the provider, the model, the prompt, and every output-affecting parameter (system, temperature, maxTokens, output), so a changed prompt or model never returns a stale answer. apiKey is never part of the key.

What you see. A cache-served step shows a cache pill in the run inspector carrying hit, key, and ageMs, with zero tokens recorded. The console's AI view has a cache hit rate for the window.

fallback - survive a rate-limited model

A fallback chain keeps one call alive when a model is rate-limited or down. The primary model is tried first; a retryable failure (429, a 5xx, or a timeout) advances to the next candidate, and the first to return wins. Its result is the step's durable output, so the caller never sees the failover.

const answer = await ctx.step.ai.generate("answer", {
  model: "claude-opus-4-8",
  prompt: question,
  fallback: [{ model: "claude-sonnet-4-6" }, { model: "claude-haiku-4-5" }],
});

from duraton import FallbackCandidate, GenerateOptions

answer = await ctx.step.ai.generate("answer", GenerateOptions(
    model="claude-opus-4-8",
    prompt=question,
    fallback=[FallbackCandidate(model="claude-sonnet-4-6"), FallbackCandidate(model="claude-haiku-4-5")],
))

answer, err := duraton.Generate(c, "answer", duraton.GenerateOptions{
	Model:    "claude-opus-4-8",
	Prompt:   question,
	Fallback: []duraton.FallbackCandidate{{Model: "claude-sonnet-4-6"}, {Model: "claude-haiku-4-5"}},
})

Only 429, 5xx, and timeout advance the chain. A terminal 4xx (a malformed request, an auth failure) fails the step immediately - another model will not fix a bad request - and an exhausted chain fails the step too, re-throwing the last error so the workflow's own retry policy still applies. Falling back to a cheaper model changes what the call costs, so a chain interacts with cap and budget through whichever model actually served.

What you see. The step shows a chain pill carrying chain (the models tried, in order), used (the one that served), and reason (why the chain advanced, e.g. "claude-opus-4-8: 429").

Watching spend

The console's Budgets view is the single place these controls surface together: every workflow's configured budget, tokenThrottle, and cap with a live meter of its window spend against the ceiling (amber at warnAtPct), plus the inference cache's hit rate. It is read-only - a control is declared in workflow code and enforced by Duraton, so there is nothing to create there.

For totals rather than ceilings, the AI view carries the window's spend, tokens, average latency, and cache-hit rate, broken down by hour, by model, and by workflow. An agent reads the same numbers through the ai_spend MCP tool.