Kafka connector

The Kafka connector is a standalone process you run next to your own Kafka. It joins a consumer group, reads the topics you name, and sends each record to Duraton as an event. Your workflows stop being Kafka consumers; they become event handlers with durable state, retries, and a run history.

If your broker is reachable from Duraton, you can skip the process entirely: a native source is the same consumer configured in your workspace, and every page here applies to it unchanged.

Duraton is only ever a consumer. It never produces, never owns your topics, and never becomes a dependency of your cluster.

Start here

The boundary

Duraton is not a broker and not a stream processor. If you need to reprocess last Tuesday from offset zero, that is Kafka. If you need the work triggered by that message to actually finish, that is Duraton.

Concretely, the connector does not:

• implement the Kafka wire protocol as a server, or accept produces
• own your partitions, your offsets, or your retention
• own rewind
• run a Schema Registry
• do stream processing: no joins, no windows, no aggregation

Not owning those does not mean not exposing them. The connector is an ordinary consumer group member with ordinary committed offsets, so rewind is still yours: kafka-consumer-groups --reset-offsets against the group id you configured does what it always did, and kafka-consumer-groups --describe shows lag the way it does for any other consumer.

The dead-letter topic goes away

A record that Duraton accepts becomes a run. If the work fails, you get a failed run with its terminal error and its full step history, replayable from the console - not an opaque payload on a side topic waiting for someone to write a drain consumer. The one thing a dead-letter topic still covers is a record Duraton will never accept at all, and that is the poison policy, which can write one if you want it. What goes away is the dead-letter topic as the default home for failure.

Shape of the deployment

  your producers  -->  your Kafka topics  -->  connector  -->  POST /events  -->  durable runs
                              ^                     |
                              +--- commit offset ---+
                                 (only once Duraton has accepted the event)

One connector process is one consumer group member. Run more replicas of the same group.id and Kafka spreads the partitions across them, exactly as it would for any consumer you wrote yourself. Nothing about the connector is stateful: the offset in Kafka is the whole of its durable state.

Next

• Quickstart - a record to a run in four commands.
• Native sources - the same consumer without the process, and how to migrate.
• Configuration - the two blocks, and what a record becomes.
• Delivery - when the offset moves, and what happens when it cannot.