panic: interface conversion: interface {} is nil, not chan *sarama.ProducerError

时间:2023-03-08 18:42:57

使用golang kafka sarama 包时,遇到如下问题:

高并发情况下使用同步sync producer,偶尔遇到crash:

panic: interface conversion: interface {} is nil, not chan *sarama.ProducerError

goroutine 413 [running]:

github.com/Shopify/sarama.(*syncProducer).handleSuccesses(0xc420384840)

/home/lanyang/github.com/Shopify/sarama/sync_producer.go:147 +0x139

github.com/Shopify/sarama.handleSuccesses)-fm()

/home/lanyang/src/github.com/Shopify/sarama/sync_producer.go:76 +0x2f

github.com/Shopify/sarama.withRecover(0xc4201d0750)

/home/lanyang/src/github.com/Shopify/sarama/utils.go:43 +0x45

created by github.com/Shopify/sarama.newSyncProducerFromAsyncProducer

/home/lanyang/src/github.com/Shopify/sarama/sync_producer.go:76 +0x112

问题不容易复现,github上已经留言:

https://github.com/Shopify/sarama/issues/785

download

https://github.com/Shopify/sarama

https://shopify.github.io/sarama/

doc

https://godoc.org/github.com/Shopify/sarama

依赖

github.com/davecgh/go-spew

github.com/eapache/go-resiliency

github.com/eapache/go-xerial-snappy

github.com/eapache/queue

github.com/pierrec/lz4

github.com/pierrec/xxHash

github.com/rcrowley/go-metrics

github.com/golang/snappy

如果kafka down掉,又恢复,client 可自动连接上。

异步方式,网络断开后,仍然会从queue,放入producer的channel中,重连后,这些内容丢失。

例子参考

sync

http://legendtkl.com/2016/06/30/kafka-in-action/

https://studygolang.com/articles/7102

async

https://studygolang.com/articles/11199