AI steps

step.ai makes a model call a durable step. Like step.run, each call takes a stable id, records its result under that id, and returns the saved result on replay instead of calling the model again. So a retry after a crash never re-spends on work that already completed.

New to AI steps? The AI quickstart walks you from a first generate call to spend landing in the console.

Duraton stores the AI metadata (model, token counts, latency) as an opaque journal block. It never parses it and never stores your prompt, the response text, or your API key - those stay in your runner.

const { output } = await ctx.step.ai.generate<Triage>("classify", {
  model: "claude-opus-4-8",
  prompt: `Classify this ticket: ${subject}`,
  output: triageSchema,
});

The step.ai API

step.ai.generate

Make one model call as a durable step. The built-in provider is Anthropic; the request is validated, sent, and the result memoized under id.

const result = await ctx.step.ai.generate("draft-reply", {
  model: "claude-opus-4-8",
  prompt: `Write a one-line apology for ticket ${ticketId}.`,
});
// result.text, result.model, result.usage.inputTokens, result.usage.outputTokens

generate returns a StructuredResult<T>:

Structured output and durable re-ask

Pass output (a JSON Schema) to constrain the model and get a typed, validated value back on result.output. If the response fails to parse or validate, generate re-prompts with the validation error - each re-ask is its own memoized step, so the retry survives a crash and never repeats a committed attempt. Add validate for semantic rules the schema can't express.

const { output } = await ctx.step.ai.generate<{ category: string; priority: string }>("triage", {
  model: "claude-opus-4-8",
  prompt: `Triage: ${subject}`,
  output: {
    type: "object",
    properties: { category: { type: "string" }, priority: { type: "string" } },
    required: ["category", "priority"],
  },
  reask: 2,
  validate: (v) => (["low", "normal", "high"].includes((v as { priority: string }).priority) ? undefined : "priority out of range"),
});

Streaming

Pass stream: true to feed the model's tokens to a live viewer as they arrive. Each delta is appended to the run's durable timeline as an ai_chunk frame, so a viewer sees the text build in real time and a late or reconnecting viewer replays it from token 0. The return value is unchanged - result.text is still the complete response, memoized on replay - so streaming affects only what a viewer sees while the step runs.

const result = await ctx.step.ai.generate("summarize-thread", {
  model: "claude-opus-4-8",
  prompt: `Summarize this thread:\n\n${thread}`,
  stream: true,
});
// result.text is the full summary; deltas streamed live on the way there.

Live deltas need a live channel to Duraton, which the connect runner transport provides; over an HTTP serve runner the call falls back to a plain generate (identical result, only the final text recorded). See the Streaming concept for the timeline frames, resumability, and the useStream React hook.

Fallback chains

Pass fallback - an ordered list of backup models - to keep a call resilient when a model is rate-limited or down. The primary model is tried first; if it fails with a retryable error (429, a 5xx, or a timeout), the call advances to the next candidate, and the first one to return wins. Its result is the step's durable output, so a caller never sees the failover.

const answer = await ctx.step.ai.generate("answer", {
  model: "claude-opus-4-8",
  prompt: question,
  fallback: [{ model: "claude-sonnet-4-6" }, { model: "claude-haiku-4-5" }],
});

Each candidate is { model, provider? }; provider defaults to the call's provider, so a chain can span providers once you have more than one adapter configured. The step's journal records the outcome:

The console renders this as a chain pill on the AI step, and it rides the opaque journal so an agent reading the run over MCP sees the same chain / used / reason.

Inference cache

Set cache to reuse the result of an identical earlier call instead of paying for it again. On a hit the provider is never called, so the step commits with zero spend - the cache is the one control that reduces spend rather than capping it, and a cached call counts nothing against cap / budget / tokenThrottle. Where step memoization already makes a replay free, the cache makes an identical call in a different run free too.

const answer = await ctx.step.ai.generate("answer", {
  model: "claude-opus-4-8",
  prompt: question,
  temperature: 0,   // required: caching engages only for a deterministic call
  cache: true,      // or { ttlMs, seed }
});

The key is an exact match over the runner's app, the model, the prompt, and every output-affecting parameter, so no entry ever crosses a project boundary and a different config never returns a stale answer. The step's journal records the outcome:

The console renders this as a cache pill on the AI step, and it rides the opaque journal so an agent reading the run over MCP sees the same hit / key / ageMs.

step.ai.wrap

Makes a model call you already write yourself - through the OpenAI SDK, the Anthropic SDK, the Vercel AI SDK, or anything else - a durable step, with no other change to the call site. wrap returns your function's value unchanged; when it recognizes the response shape it enriches the journal with the model and token counts and records which library it wrapped.

import OpenAI from "openai";
const openai = new OpenAI();

const completion = await ctx.step.ai.wrap("classify", () =>
  openai.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: subject }],
  }),
);

Recognized shapes: the Anthropic SDK, the OpenAI SDK, and the Vercel AI SDK. An unrecognized value still becomes a durable wrap step - you just get less metadata on the journal.

step.ai.embed

Turn a list of inputs into vectors, one durable batch at a time. Anthropic has no embeddings API, so you supply the embedding call (embed); Duraton owns the batching and per-batch checkpointing. If a batch fails, only that batch re-runs on retry - committed batches are not re-embedded.

const { vectors } = await ctx.step.ai.embed("embed-kb", {
  model: "voyage-3",
  inputs: ["duplicate charge policy", "annual plan refunds", "refund SLA"],
  embed: (batch) => voyage.embed(batch),
  batchSize: 2,
});

embed returns { vectors } - one vector per input, in input order.

step.ai.loop

A durable agent loop. Each turn is your own model call (bring-your-own, normalized to tool calls or a final answer); the loop executes the tools the turn requested and feeds the results into the next turn, until the model returns a final answer, stop fires, or maxIterations is reached.

Every turn is a durable step (id:iter:N) and every tool call is a durable step (id:iter:N:tool:<callId>), so an agent that crashes mid-run resumes at the last committed turn.

const agent = await ctx.step.ai.loop<{ resolution: string }>("agent", {
  prompt: `Resolve the ticket about: ${subject}`,
  maxIterations: 6,
  tools: {
    "search-kb": { handler: (input) => searchKb(input) },
    "lookup-order": { workflow: "orders.lookup", app: "orders" },
  },
  turn: (ctx, iteration) => callModel(ctx.prompt, ctx.history, iteration),
});
// agent.final, agent.iterations, agent.stopReason

Your turn returns a LoopTurn - either tool calls to run, or a final answer:

A tool is either a handler (a local function) or a workflow (another Duraton workflow, called as a linked child run):

maxIterations bounds how many turns a loop may run; a per-run budget cap bounds how much it may spend. When a run reaches its cap the loop halts before its next turn's model call and the run fails with a BudgetError - the committed turns stay, and the halted turn is the loop's last (failed) iteration.

ctx.history gives each turn the prior turns' toolCalls and toolResults, so your model call can see what it has already tried. loop returns:

Providers

step.ai.generate resolves its provider name to an AIProvider adapter through a port, so you can supply your own instead of the built-in registry. Pass resolveProvider to connect or serve and every generate call in that runner goes through it - the call sites are unchanged.

import { connect } from "@duraton/sdk";
import { type AIProvider, createAnthropicProvider, getProvider } from "@duraton/sdk/ai";

const recording: AIProvider = {
  name: "anthropic",
  generate: (req) => fixtures[req.prompt] ?? getProvider("anthropic").generate(req),
};

connect({
  app: "support-app",
  workflows,
  resolveProvider: (name) => (name === "anthropic" ? recording : getProvider(name)),
});

An AIProvider implements generate(req) and, optionally, stream(req, onDelta) (an adapter without it falls back to generate, so stream: true still returns the right text) and classifyError(err) (which decides whether a failure is retryable, and so whether a fallback chain advances - an unclassified error is treated as terminal).

Cost

Duraton holds no model price list, so a call's cost is absent unless your runner supplies it. Pass resolveCost - the CostSource port - and each step.ai call is priced from the axes the journal already holds. Supplying it is what makes cap: { maxCost } and budget: { maxCost } bite; maxTokens needs nothing, because tokens are metered from every call.

import type { CostSource } from "@duraton/sdk/ai";

const PRICES: Record<string, { in: number; out: number }> = {
  "claude-opus-4-8": { in: 5 / 1_000_000, out: 25 / 1_000_000 },
};

const resolveCost: CostSource = ({ model, tokensIn = 0, tokensOut = 0 }) => {
  const p = model ? PRICES[model] : undefined;
  return p ? tokensIn * p.in + tokensOut * p.out : undefined;   // undefined leaves cost absent
};

connect({ app: "support-app", workflows, resolveCost });

Returning undefined leaves the cost absent - Duraton never fabricates a zero - and a call that already carries an explicit cost is left untouched.

Cache store

The inference cache is backed by the AICache port, so the store is swappable. The default is createMemoryCache(): a process-local Map with per-entry TTL and LRU eviction, bounded at 1000 entries. Pass cache to connect or serve to swap it - for a store shared across runner processes, say.

import { createMemoryCache } from "@duraton/sdk/ai";

connect({
  app: "support-app",
  workflows,
  cache: createMemoryCache({ maxEntries: 10_000 }),
});

The store is only ever consulted for a call that opted in with cache - its mere presence changes nothing. The cached completion is held runner-side: Duraton's journal records only the cache metadata (hit, key, ageMs), never the payload. A store you share across processes must seed its keys deliberately, since the default seed (the runner's app) assumes the process boundary isolates it.

Workflows as tools

Hand any workflow to a loop as a tool and the model can drive it: the tool call becomes a linked child run, the model's tool input is the child's trigger data, and the child's result is the tool result. The child is a full durable run of its own - it can retry, sleep, and call further workflows - and shows up linked to the parent in the inspector. This is how the orchestrator-workers pattern maps onto Duraton.

Durable steps and replay

Because every step.ai call is a durable step, the same replay rules as regular steps apply: keep turn, validate, and stop deterministic (a pure function of their inputs), since they run again on replay while the memoized model results do not. Model calls happen exactly once per committed step; everything around them must be replay-safe.

For the concepts behind durable AI steps and end-to-end recipes, see AI agents.