Kafka Service
The “Kafka Service” plugin is no longer supported.
This SCCM Manager plugin is used to configure the Kafka Message Service. It also allows you to view status information.
The service is controlled using the Start, Stop, and Restart buttons. The drop-down menu next to them allows you to select specific configurations.
Kafka message
The Kafka messages read by the producer from the producer inbox follow the following naming convention: topicname_id.json
The topic name and ID must always be separated by an underscore, but the topic name may still contain underscores.
Service configuration
To ensure that the Kafka service works properly, several configuration steps must be performed. The most important configurations are listed here.
For more information, visit the official Apache Kafka documentation
| Property | Description |
| Enable Consumer | Enables the Consumer Service. |
| Enable Producer | Enables the Producer Service. |
| Consumer Config Instance | Selected configuration instance (read-only) |
| Consumer Topic | Sets the theme for receiving messages. |
| Producer Config Instance | Selected configuration instance (read-only) |
| Producer File Types | Specifies the file types used to identify the messages to be sent. Separate multiple types with a pipe. Example: json|xml |
| Producer Path | Sets the inbox path for new messages to be sent. |
| Producer Processing Max Retieres | Specify the maximum number of retry attempts in the event of an error. |
| Producer Processing Retry Timeout | Set the timeout for retries when an error occurs. |
Consumer configuration
| Property | Description |
| Api Version Request |
Retrieves the broker's supported API versions to adapt the functionality to the available protocol features. If set to “false,” or if the API version request fails, the fallback version `broker.version.fallback` is used.
|
| Auto Offset Reset |
Action to be performed if there is no initial offset in the offset memory or if the desired offset is outside the range: 'smallest',‘earliest’ - automatically resets the offset to the smallest offset ‘largest’,'latest' - automatically resets the offset to the largest offset ‘error’ - triggers an error, which is detected by consuming messages and checking ‘message->err’. |
| Bootstrap Servers | Initial list of brokers as a CSV list of broker-host or host:port. The application can also use `rd_kafka_brokers_add()` to add brokers at runtime. |
| Client Id | The Client ID. |
| Enable Auto Commit | Enables/disables automatic background offset calculation. |
| Enable Auto Offset Store |
Enables/disables the storage of the offset of the last message received. The offset buffer is an internal buffer for the next offset, which is (automatically) transferred for each partition. |
| Enable Partition Eof | Enables or disables the sending of the RD_KAFKA_RESP_ERR__PARTITION_EOF message when the consumer reaches the end of a partition. |
| Groud Id | The string representing the client group ID. All clients with the same group.id belong to the same group. |
| Isolation Level | Controls how transactional messages are read: `read_committed` - returns only transactional messages that have been committed. `read_uncommitted` - returns all messages, including transactional messages that have been rolled back. |
| Max Poll Interval Ms | The maximum allowed time between calls to consume messages for high-level consumers. If this interval is exceeded, the consumer is considered to have failed, and the group is rebalanced to reassign the partitions to another member of the consumer group. |
| Secruity Protocol | A protocol used for communicating with brokers. |
| Session Timeout Ms |
Timeout for client group sessions and error detection. The consumer sends periodic heartbeats (heartbeat.interval.ms) to indicate to the broker that it is operational. If the broker does not receive any heartbeats from a group member within the session timeout period, the broker removes the consumer from the group and triggers a resynchronization. |
| Statistics Interval Ms |
librdkafka Statistics - Message send interval. A value of 0 disables sending. |
Producer configuration
| Property | Description |
| Api Version Request |
Retrieves the broker's supported API versions to adapt the functionality to the available protocol features. If set to “false,” or if the API version request fails, the fallback version `broker.version.fallback` is used. |
| Bootstrap Servers | Initial list of brokers as a CSV list of broker-host or host:port. The application can also use `rd_kafka_brokers_add()` to add brokers at runtime. |
| Client Id | The Client ID. |
| Enable Idempotence | Enables/disables the feature that ensures messages are successfully generated exactly once and in the original order in which they were created. |
| Linger Ms |
The wait time, in milliseconds, before messages in the producer queue are accumulated and sent to the broker as message sets. A higher value allows for the accumulation of larger and more efficient (less overhead, improved compression) message sets at the cost of increased latency in message delivery. |
| Message Send Max Retires | Number of retry attempts when sending a failed message. |
| Message Timeout Ms | Local timeout for messages. This value applies only locally and limits the amount of time a message waits for successful delivery. If the value is 0, the wait time is unlimited. A delivery error occurs if either the number of retry attempts or the message timeout is exceeded. |
| Partitioner |
`random` - random distribution `consistent` - CRC32 hash of key (Empty and NULL keys are assigned to a single partition) `consistent_random` - CRC32 hash of key (Empty and NULL keys are randomly partitioned) `murmur2` - Java Producer-compatible Murmur2 hash of key (NULL keys are assigned to a single partition) `murmur2_random` - Java Producer-compatible Murmur2 hash of key (NULL keys are randomly partitioned. This is functionally equivalent to the default partitioner in the Java Producer.) |
| Queue Buffering Max Kbytes | The maximum total message size allowed in the producer queue. This queue is shared by all topics and partitions. This property takes precedence over `queue.buffering.max.messages`. |
| Queue Buffering Max Messages | Maximum number of messages allowed in the producer queue. This queue is shared by all topics and partitions. |
| Security Protocol | A protocol used for communicating with brokers. |
| Statistics Interval Ms | librdkafka Statistics - Message send interval. A value of 0 disables sending. |