Kafka Service
This SCCM Manager plug-in is used to configure the Kafka Message Service. It also allows viewing the status information.
The service is controlled with the Start, Stop and Restart buttons. The individual configurations can be selected via the combo box next to it.
This video shows the basic configuration of the Kafka service:
Video fehlt hier: kafka_service_first_start
Kafka Message
The Kafka messages that are read by the Producer from the Producer Inbox have the following naming conventions: topicname_id.json
Topicname and Id must always be separated with an underscore, the topicname may still contain underscores.
Service configuration
In order for the Kafka service to be functional, some configurations must be made. The most important configurations are listed here.
Für mehr Informationen besuchen Sie die offizielle Apache Kafka Dokumentation
| Property | Description |
| Enable Consumer | Activates the consumer service. |
| Enable Producer | Activates the producer service. |
| Consumer Config Instance | Selected configuration instance (read only) |
| Consumer Topic | Sets the subject for receiving messages. |
| Producer Config Instance | Selected configuration instance (read only) |
| Producer File Types | Sets the file types to identify the messages to be sent. Separate multiple types with pipe. Example: json|xml |
| Producer Path | Sets the inbox path for new messages to be sent. |
| Producer Processing Max Retries | Set the maximum retries when an error occurs. |
| Producer Processing Retry Timeout | Set the retry time limit when an error occurs. |
Consumer configuration
| Property | Description |
| Api Version Request |
Requests the supported API versions of the broker to match the functionality to the available protocol features. If set to false, or if the api version request fails, the fallback version `broker.version.fallback` will be used. |
| Auto Offset Reset |
Action to be performed if there is no initial offset in the offset memory or the desired offset is out of range: 'smallest','earliest' - automatically resets the offset to the smallest offset. 'largest','latest' - resets the offset automatically to the largest offset 'error' - triggers an error determined by consuming messages and checking 'message->err'. |
| Bootstrap Servers | Initial list of brokers as CSV list from broker host or host:port. The application can also use `rd_kafka_brokers_add()` to add the broker at runtime. |
| Client Id | The Client-ID. |
| Enable Auto Commit | Enables/disables automatic setting of offsets in the background. |
| Enable Auto Offset Store |
Enables/disables the storage of the offset of the last message received. The offset memory is an internal memory for the next offset that is transmitted (automatically) for each partition. |
| Enable Partition Eof | Enables/disables sending the RD_KAFKA_RESP_ERR__PARTITION_EOF message when the consumer reaches the end of a partition. |
| Groud Id | String of the client group ID. All clients with the same group.id belong to the same group. |
| Isolation Level | Controls how transactionally written messages should be read: `read_committed` - returns only transactional messages that have been committed. `read_uncommitted` - returns all messages, including transactional messages that have been aborted. |
| Max Poll Interval Ms | Maximum time allowed between calls to consume messages for high-level consumers. If this interval is exceeded, the consumer is considered failed and the group is rebalanced to reassign the partitions to another member of the consumer group. |
| Secruity Protocol | Protocol used for communication with brokers. |
| Session Timeout Ms |
Timeout for client group sessions and error detection. The Consumer sends periodic heartbeats (heartbeat.interval.ms) to indicate to the Broker that it is ready for operation. If the broker does not receive heartbeats for a group member within the session timeout, the broker removes the consumer from the group and triggers a rebalance. |
| Statistics Interval Ms |
librdkafka statistics -message send interval. The value 0 disables sending. |
Producer configuration
| Property | Description |
| API Version Request |
Requests the supported API versions of the broker to match the functionality to the available protocol features. If set to false, or the api version request fails, the fallback version `broker.version.fallback` is used. |
| Bootstrap Servers | Initial list of brokers as CSV list from broker host or host:port. The application can also use `rd_kafka_brokers_add()` to add the broker at runtime. |
| Client ID | The Client-ID. |
| Enable Idempotence | Enables/disables messages to be successfully created exactly once and in the original creation order. |
| Linger ms |
Waiting time in milliseconds for messages to accumulate in the Producer queue and be sent to the Broker den as message batches (MessageSets). A higher value allows the accumulation of larger and more effective (less overhead, improved compression) message batches at the cost of increased message delivery latency. |
| Message Send Max Retires | Number of retries when sending a failed message. |
| Message Timeout Ms | Local message timeout. This value applies only locally and limits the time a produced message waits for successful delivery. With the value 0, the waiting time is unlimited. A delivery error occurs when either the number of retries or the message timeout is exceeded. |
| Partitioner |
random` - random distribution `consistent` - CRC32 hash of key (Empty and NULL keys are assigned to a single partition) `consistent_random` - CRC32 hash of key (Empty and NULL keys are partitioned randomly) `murmur2` - Java Producer compatible Murmur2 hash of key (NULL keys are assigned to a single partition) `murmur2_random` - Java Producer compatible Murmur2 hash of key (NULL keys are partitioned randomly. This is functionally equivalent to the default partitioner in Java Producer). |
| Queue Buffering Max kBytes | Maximum total message size allowed in the Producer queue. This queue is shared by all topics and partitions. This property has a higher priority than queue.buffering.max.messages. |
| Queue Buffering Max Messages | Maximum number of messages allowed in the Producer queue. This queue is shared by all topics and partitions. |
| Security Protocol | Protocol used for communication with brokers. |
| Statistics Interval ms | librdkafka statistics -message send interval. The value 0 disables sending. |