Command line

Drive the connector from a terminal: the four duraton-kafka commands, the flags and exit codes of consume, and what it logs while running.

The Kafka connector ships as one binary with four commands. Two of them reach nothing, which is what makes them safe to run against a cluster that matters.

Four commands, and the whole surface is on help:

duraton-kafka help

duraton-kafka consumes Kafka topics and sends each record to Duraton as an event.

Usage:
  duraton-kafka <command> [flags]

Commands:
  consume    consume the configured topics and send each record to Duraton
  dry-run    read a few records and print the events they would become, sending nothing
  validate   check a configuration file and report what it resolves to
  version    print the connector version

Run duraton-kafka <command> --help for a command's flags.

Command	What it does	Reaches
`consume`	Joins the group, reads the configured topics, sends each record to Duraton.	The cluster and Duraton.
`dry-run`	Reads a few records and prints what each one would become.	The cluster. Nothing reaches Duraton.
`validate`	Checks a file and prints what it resolves to.	Nothing.
`version`	Prints the connector version.	Nothing.

validate and dry-run have a page of their own: Diagnostics.

`consume`

DURATON_URL=https://api.duraton.dev \
DURATON_APP=billing-ingress \
DURATON_API_KEY=dtn_live_... \
duraton-kafka consume --config kafka.yaml --listen :9464

Usage of consume:
  -allow-unknown-kafka-properties
    	warn about an unrecognised Kafka property instead of refusing to start
  -config string
    	path to the configuration file
  -listen string
    	address to serve /healthz, /readyz and /metrics on, for example :9464
  -log-level string
    	one of debug, info, warn, error; also read from DURATON_KAFKA_LOG_LEVEL (default "info")

Flag	Type	Default	Bounds and notes
`--config`	file path	-	Required. Same flag on `validate`, so checking a file and running with it are spelled identically.
`--listen`	host:port	unset	Opens `/healthz`, `/readyz` and `/metrics` on one address. While it is unset no port is opened and no metric is collected: the connector runs inside someone else's network, so it opens a port only when asked.
`--allow-unknown-kafka-properties`	boolean	`false`	Downgrades an unrecognised `kafka:` property from an error to a warning. Also settable as `DURATON_KAFKA_ALLOW_UNKNOWN`.
`--log-level`	enum	`info`	`debug`, `info`, `warn`, `error`. `debug` adds a line per record - accepted with its run id, filtered - and the full resolved configuration at start-up. Tombstones are the exception: they are tallied and reported one line per partition per batch, because a compacted topic is mostly tombstones and a line each would bury every other line. At `info` you get lifecycle, warnings, stalls and poison only. Also settable as `DURATON_KAFKA_LOG_LEVEL`, so a running deployment can be turned up without rewriting its entrypoint; the flag wins when both are given.

One dash or two are both accepted. Logs are JSON, one object per line, on stderr; an attribute whose name looks like a credential (key, apiKey, sasl.password, client_secret, authorization, access-token, cookie) is written redacted rather than dropped.

Exit code	Meaning
`0`	Stopped by `SIGINT` or `SIGTERM` after draining, a valid file under `validate`, or at least one record read under `dry-run`.
`1`	A configuration or start-up fault, or a halt: a rejected API key, `onPoison: halt` reaching a record, or `dry-run` reaching `--wait` with nothing read.
`2`	A usage fault: no command, an unknown command, a missing `--config`, an unknown log level, or `--from` given with `duraton.assign`.

`version`

version prints the version of the binary in front of you, and the same string is the version attribute on the start-up log line - so a deployment that is already running reports its own version without being restarted or exec'd into.

duraton-kafka version

duraton-kafka 1.0.0

{"time":"...","level":"INFO","msg":"the connector is starting","version":"1.0.0","app":"billing-ingress","destination":"https://api.duraton.dev","decoder":"json","envelope":"wrapped"}

A build nobody released - one you built from the tree yourself - reports (devel) rather than claiming a version that was never published.

Which brokers

Broker versions are neither configured nor pinned. The connector asks the cluster which protocol versions it offers and uses what comes back, which is why the three pinning properties are refused rather than accepted and ignored. It speaks the classic consumer rebalance protocol and only that one, so a broker that also offers the next-generation protocol is not a special case and the group timeouts stay live.

Cluster	Verified against
Apache Kafka	4.3.1, KRaft mode. Every transport the Security page documents - `PLAINTEXT`, `SSL`, mTLS, `SASL_PLAINTEXT`, `SASL_SSL` - is exercised against it on every build, each with its negative case.
Redpanda	26.2.1, the broker in the local stack and the one the consume loop is developed against.

Those two are what is actually tested, and no minimum broker version is enforced or claimed beyond them. Automatic negotiation is what every broker from 0.10.0 onwards offers, which is the point below which pinning would have mattered - but an older cluster than the two above is untested rather than supported.

Diagnostics - validate and dry-run in full.
Deploying - the same commands in a container.
Observability - what --listen opens.

`consume`

`version`

Which brokers

Next

On this page