I'm using librdkafka's c++ client and Confluent's dotnet client to consume messages. The dotnet version consumes all of the records for a given topic and partition but the c++ client only consumes at most 10 records for the same topic and partition. I've played around with the properties given here https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md so that both versions have the same configuration but nothing has worked yet. Does anyone know why this would happen?
我正在使用librdkafka的c ++客户端和Confluent的dotnet客户端来使用消息。 dotnet版本使用给定主题和分区的所有记录,但c ++客户端仅消耗同一主题和分区的最多10条记录。我已经玩过这里给出的属性https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md,这样两个版本都有相同的配置,但还没有任何工作。有谁知道为什么会这样?
1 个解决方案
#1
0
You are not specifying -X topic.auto.offset.reset=earliest (defaults to latest offset) so your consumer will start consuming from the end of the partition, and with no new messages coming in your consumer will not see any messages. During testing also make sure to use a unique group.id for each run to avoid resuming from previously committed offsets.
您没有指定-X topic.auto.offset.reset = earliest(默认为最新偏移量),因此您的消费者将从分区的末尾开始消费,并且消费者中没有新消息将看不到任何消息。在测试期间,还要确保为每次运行使用唯一的group.id,以避免从先前提交的偏移量恢复。
#1
0
You are not specifying -X topic.auto.offset.reset=earliest (defaults to latest offset) so your consumer will start consuming from the end of the partition, and with no new messages coming in your consumer will not see any messages. During testing also make sure to use a unique group.id for each run to avoid resuming from previously committed offsets.
您没有指定-X topic.auto.offset.reset = earliest(默认为最新偏移量),因此您的消费者将从分区的末尾开始消费,并且消费者中没有新消息将看不到任何消息。在测试期间,还要确保为每次运行使用唯一的group.id,以避免从先前提交的偏移量恢复。