Kafka connector

Reference: the duraton: block

Every key in the duraton block of the connector configuration, with its type, default and bounds - subscription, decoding, the envelope, the poison policy, retry windows, and the event mapping.

Duraton's own half of the connector configuration: everything about what a record becomes. The other half keeps Apache Kafka's property names and is in the kafka: reference.

Every bound below is enforced at startup. validate prints the resolved value of each one.

KeyTypeDefaultNotes
urlstringDURATON_URLRequired. An http or https address.
keysecret-Never written here. DURATON_API_KEY or keyFile.
keyFilefile path-A file holding the API key.
appstringDURATON_APPRequired. At most 256 characters.
topicslist of strings-Exact names. Excludes topicPattern and assign.
topicPatternregular expression-Held to the whole topic name.
excludeTopicslist of strings-Requires topicPattern.
assignmap of topic to partition and offset-Excludes topics and kafka.group.id. Partition and offset are both non-negative.
decoderenumjsonjson, raw, text, avro-confluent, json-schema-confluent.
envelopeenumwrappedwrapped, value-only.
filterCEL expression-Empty means no filtering.
onNullValueenumskipskip, deliver, poison.
onPoisonenumcommit-and-logcommit-and-log, halt, commit-and-produce.
poisonTopicstring-Required by onPoison: commit-and-produce, and refused under any other policy. Cannot be a topic this connector consumes.
commit.intervalduration1sLongest an accepted record waits before its offset is committed. There is no batch-size key: the batch is kafka.max.poll.records.
retry.minduration250msFloor of the wait between attempts at a record the destination has not accepted. Longer than zero.
retry.maxduration30sCeiling of that wait. Longer than zero, and not shorter than retry.min.
mapping.eventNameliteral or paththe topic nameThe event name.
mapping.appliteral or pathduraton.app
mapping.targetAppliteral or path-Route to one app's triggers.
mapping.runnerliteral or path-Pin runs to one runner.
mapping.sessionliteral or path-Thread runs into one conversation.
mapping.tagsmap of tag name to pathnoneAt most 20. Names are at most 64 characters of letters, digits, _, . and -.
mapping.dedupeIdtemplate{topic}:{partition}:{offset}Placeholders are topic, partition, offset, key, or a dotted path. At most 256 characters.
schemaRegistry.urlstring-An http or https address. Required by avro-confluent, and by protobuf-confluent, which is not built yet. json-schema-confluent does not need it.
schemaRegistry.authsecret-Never written here. KAFKA_SCHEMA_REGISTRY_AUTH or authFile. The API key and secret joined by a colon, as basic.auth.user.info takes them; sent as HTTP basic auth.
schemaRegistry.authFilefile path-
schemaRegistry.ttlduration5mHow long a resolved schema is held before it is fetched again. Longer than zero.
sasl.tokenProviderenumoidc when a token endpoint URL is setoidc, static.
sasl.tokenFilefile path-A file holding an OAUTHBEARER token.

Every mapped string field is bounded at 256 characters, matching what the event endpoint accepts.

Next

On this page