Kafka inbound
Overview
A task can be set up to connect to a Kafka topic, and continuously consume messages. When a job for this task is submitted, it runs indefinitely until it is manually stopped from the Task Scheduler.
In this setup, the producer class acts as the Kafka consumer, retrieving messages as they become available on the topic. Once a message is consumed, the producer places it in a Hazelcast distributed queue for processing by any available DIH consumer thread.
Parameters
| Parameter | Description |
|---|---|
prop:bootstrap.servers |
Comma-separated list of Kafka bootstrap servers used to connect to the Kafka cluster. |
csvHeader |
Comma-separated list of column names for CSV messages consumed from Kafka. Required when dataFormat is set to CSV. |
dataFormat |
Format of the Kafka message payload. Supported values are CSV and RAW. |
tableName |
Name of the destination table where Kafka data will be stored. |
prop:topic |
Name of the Kafka topic to consume messages from. |
Examples
Executable
Task
