Diagnostics

Two commands answer the two questions that used to need a production incident: what is this process actually configured to do, and what do my records turn into. Neither sends an event, and validate contacts nothing at all.

Both take the same --config file that consume takes.

Checking a configuration

validate loads the file, applies every default, resolves every secret source, runs every cross-property rule, and prints the resolved configuration with secrets redacted. It contacts no broker and no Duraton endpoint, so it is safe to run anywhere, and its output is safe to paste into a support thread.

duraton-kafka validate --config kafka.yaml

the configuration is valid; it resolves to:
  duraton.url=https://api.duraton.dev
  duraton.key=<redacted>
  duraton.app=billing-ingress
  duraton.topics=payments.events
  duraton.decoder=json
  duraton.envelope=wrapped
  duraton.onNullValue=skip
  duraton.onPoison=commit-and-log
  duraton.commit.interval=1s
  duraton.retry.min=250ms
  duraton.retry.max=30s
  duraton.schemaRegistry.ttl=5m0s
  duraton.mapping.app=billing-ingress
  duraton.mapping.dedupeId={topic}:{partition}:{offset}
  kafka.bootstrap.servers=kafka-1:9092,kafka-2:9092
  kafka.client.id=duraton-kafka-ingress
  kafka.request.timeout.ms=10000
  kafka.socket.connection.setup.timeout.ms=10000
  kafka.connections.max.idle.ms=30000
  kafka.metadata.max.age.ms=300000
  kafka.retry.backoff.ms=100
  kafka.retry.backoff.max.ms=1000
  kafka.receive.message.max.bytes=104857600
  kafka.allow.auto.create.topics=false
  kafka.enable.metrics.push=true
  kafka.security.protocol=PLAINTEXT
  kafka.enable.ssl.certificate.verification=true
  kafka.ssl.endpoint.identification.algorithm=https
  kafka.ssl.protocol=TLSv1.2
  kafka.auto.offset.reset=latest
  kafka.isolation.level=read_committed
  kafka.check.crcs=true
  kafka.fetch.min.bytes=1
  kafka.fetch.max.bytes=52428800
  kafka.fetch.max.wait.ms=500
  kafka.max.partition.fetch.bytes=1048576
  kafka.max.poll.records=500
  kafka.group.id=duraton-billing
  kafka.partition.assignment.strategy=cooperative-sticky
  kafka.session.timeout.ms=45000
  kafka.heartbeat.interval.ms=3000
  kafka.max.poll.interval.ms=300000

That listing is the point of the command: every default you did not write is on it, so "what is this process actually configured to do" is never a source-reading exercise. Warnings go to stderr as JSON, one object per line; the listing goes to stdout.

What it refuses, and why, is What the connector refuses.

Seeing what a record becomes in a dry run

validate answers what the file resolves to. A dry run answers the question after it: what your actual records turn into. "Why is my app field empty" and "why is my filter excluding everything" used to be answerable only by sending real records at a real destination; this answers them in your own terminal, against the same records, without sending anything.

It reads the file consume reads and applies the same decoder, envelope, mapping, dedupe id and filter. What it prints is the request body the destination would receive, produced by the code that produces the real one - not a second description of it.

Two properties are what make it safe to point at a live topic:

• It joins no consumer group. It takes no partitions from a running connector, stores no offset, and leaves no group behind. Joining no group also means there is no committed offset to resume from, which is why --from exists and why it starts at earliest rather than honouring auto.offset.reset.
• It sends nothing. No event reaches Duraton and nothing runs.

duraton-kafka dry-run --config kafka.yaml

Output is one JSON object per record on stdout, so it reads in a terminal and pipes to jq. Each object carries topic, partition, offset, decision, and then either reason or event, which is the request body itself. The closing summary goes to stderr, so stdout stays clean.

duraton.onPoison does not apply here. A poison record is reported and reading continues whatever the policy says, because seeing every bad record in one pass is the point; halting on the first would hide the rest. That policy governs the running connector only.

The three lines below come from a configuration carrying a mapping.eventName and a filter, not from the minimal kafka.yaml in the quickstart: with that file every event would be named after its topic and nothing would be filtered.

$ duraton-kafka dry-run --config kafka.yaml --records 3
{"topic":"payments.events","partition":0,"offset":41,"decision":"send","event":{"name":"payment.captured","app":"billing-ingress","data":{"key":"A1","headers":{"content-type":"application/json"},"topic":"payments.events","partition":0,"offset":41,"timestamp":"2026-07-29T10:00:00Z","value":{"amount":4200,"type":"payment.captured"}},"dedupeId":"payments.events:0:41"}}
{"topic":"payments.events","partition":0,"offset":42,"decision":"skip","reason":"the record is not sent: the filter excluded it"}
{"topic":"payments.events","partition":1,"offset":7,"decision":"poison","reason":"the record cannot be sent: the json decoder rejected the record value: it is not valid JSON (invalid character 'o' in literal null (expecting 'u'))"}

It exits 0 when it read at least one record, and 1 on a configuration fault, a connection fault, or when nothing arrived before --wait elapsed. That last one is deliberate: a diagnostic that showed nothing is a failed diagnostic, and treating it as one is what makes the command usable in a script. A usage fault is 2 as on every command, --from with duraton.assign among them.

This is also why the connector has no payload-logging flag. Logging record payloads from a long-running process is a standing risk to whatever personal data those records carry, and a bounded, explicitly invoked command gives the same answer with none of that exposure.

How much of the topic is worth a run

One JSON object per record on stdout means the ratio you actually care about is a sort | uniq:

duraton-kafka dry-run --config kafka.yaml --records 500 2>/dev/null \
  | jq -r .decision | sort | uniq -c

   1 poison
 173 send
 326 skip

That is the number to size a deployment on, and it is worth several hundred records rather than a handful. A topic where nearly every record becomes a run needs a filter before it needs more replicas: one run per record at broker volume is the case the connector is least suited to, and a filter that rejects the uninteresting majority costs nothing per record.

2>/dev/null drops the closing summary, which goes to stderr precisely so a pipeline like this one sees only the records.

Next

• What the connector refuses - reading a rejection.
• Filtering - the expression dry-run is testing.
• Command line - every command and exit code.