Filtering

At broker volumes, one run per record does not hold. duraton.filter is the Kafka connector's way of dropping a record before it becomes anything.

Filtering, first

duraton.filter is a CEL expression evaluated before the event is sent, in the same dialect as a trigger if and with the same shape: one variable, event, carrying event.name and event.data (the envelope). Empty means no filtering.

Reach for it before you reach for anything else. Filtering here is the only free place to drop a record - the other two, the trigger predicate and the flow gates, both cost work that has already started.

Two things about the dialect decide whether an expression survives real traffic.

A filter that cannot be evaluated is a poison outcome, not a "no". CEL raises an error on a missing map key rather than answering false, so event.data.value.kind == 'sale' fails on the first record whose payload has no kind. Under the default onPoison: commit-and-log that record is logged and skipped; under onPoison: halt it stops the connector; under onPoison: commit-and-produce it is written to the poison topic. Guard every field the payload may not carry:

# fails on a record with no kind, and the record is skipped as poison
filter: 'event.data.value.kind == "sale"'

# answers false on that record, which is what you meant
filter: 'has(event.data.value.kind) && event.data.value.kind == "sale"'

Every JSON number reaches CEL as a double. The payload is parsed without a target type, so 42 arrives as 42.0: compare against 42.0 or 42 (both are doubles here), never against a value you expect to be an integer type. event.data.value.amount > 1000 behaves as written; an expression that depends on integer semantics does not.

The expression is also cost-bounded per record, so a comprehension over a large array is stopped rather than allowed to pin the consumer. That stop is a poison outcome too, with an error saying to narrow the expression.

Checking one before you ship it

dry-run applies the same filter to your real records and prints a decision of skip for each one it excluded, so an expression that excludes everything says so in your terminal rather than in production.

Next

• Delivery - what the poison policy does.
• Diagnostics - test the expression against real records.
• Triggers - the same dialect, on the workflow side.