Publishing results back

Records come in, workflows run, and their side effects are the output. That is the shape almost every Kafka connector deployment should have, and this page is mostly about why - plus the one pattern that is right when it genuinely is not enough, and the one that never is.

The default: nothing goes back

A terminal consumer is the recommended shape. The workflow charges the card, writes the row, calls the partner API, sends the mail - and stops. Nothing is published, and there is no reply topic.

  topic --> connector --> POST /events --> run --> side effects
                                                        |
                                                   (it ends here)

Reach for this until something forces you off it. It has no correlation to get wrong, no reply topic to size, no second consumer group to operate, and no partial-failure mode where the work succeeded and the announcement did not.

When another consumer needs the outcome

Systems that are already Kafka consumers are the honest exception: the enriched record belongs on a topic because that is where those consumers read. Publishing it is a step in your own workflow, using your own producer.

await ctx.step.run("publish-enriched", () =>
  producer.send({ topic: "calls.enriched", key: callId, value: enriched }),
);

await ctx.step.run("publish-enriched", lambda: producer.send("calls.enriched", key=call_id, value=enriched))

_, err := duraton.Run(c, "publish-enriched", func() (bool, error) {
	return true, producer.Send(c.Ctx, "calls.enriched", callID, enriched)
})

That one call inherits everything step.run already gives: retry with backoff if the broker is unavailable, the result recorded in step history, and the step skipped on replay once it has completed. There is no publish adapter, no sink: block and no engine feature here - the producer is yours, its configuration is yours, and Duraton's part is making the call durable.

What replay does, and does not, guarantee

A step that completed is not executed again: the run replays, and the step returns its saved result instead of calling the producer a second time. That is what makes this safe to retry.

It is not exactly-once. A crash between the broker accepting the message and Duraton recording the step's result leaves the step incomplete, and the replay publishes again. The message key is the answer, the same as anywhere else in Kafka: key by the entity so a repeat is a duplicate of a known record rather than a new one, and let compaction or an idempotent consumer absorb it.

The anti-pattern: request/response over Kafka

Do not put a caller on one topic, a durable workflow in the middle, and a reply topic on the other side, with the caller blocking on the answer.

  caller --> requests topic --> [ workflow: 30s, or an hour, or a human approval ] --> replies topic
     ^                                                                                     |
     +--------------------------------- blocked this whole time --------------------------+

The latency is unbounded by design, not by accident. A durable workflow is allowed to sleep for a day, park on a human approval, and survive a deploy in the middle - those are the features. Anything that needs a synchronous answer should not be routed through a durable execution engine at all; call the service directly, and use Duraton for the work that outlives the request.

Failures do not go back either

There is no dead-letter topic in this design, and that is a replacement rather than an omission.

A failed run keeps the terminal error and the full step history, and it is replayed from the console - with the payload edited, if the fix is in the payload. Compare that to a DLT, where you get the raw bytes back and rebuild the context yourself.

The one thing that does still go to a topic is a record the connector could never turn into an event at all - a value no decoder can read. That is the poison policy, which is about undecodable bytes, not about failed work.

Next

• Delivery - when the offset moves, and the poison policies.
• Steps - retry, memoization and the rest of the step surface.
• Behaviour worth knowing - what surprises people in production.