Issue in Kafka Importer-Version: 5.7.1-20210916-3a97ffde6c

Today we have updated the kaka version
from: OmniSci Server Version: 5.6.1-20210511-81f478b738
to: OmniSci Server Version: 5.7.1-20210916-3a97ffde6c

after update KafkaImporter module stopped working as giving core dump of 18GB. We tried multiple times and no success.

Later we replaced the “/opt/omnisci/bin/KafkaImporter” file from my old installation of 5.6.1 to new installation (version 5.7.1) and Started Working.

Summary
/opt/omnisci/bin/KafkaImporter - 5.6.1 (Working)
/opt/omnisci/bin/KafkaImporter - 5.7.1 (issue)

Kindly check.

Regards,
sumit

Hi @sumit,

Thanks for reporting us the problem.

I tried to reproduce the with little to no success.

candido@zion-legion:/opt/mapd$ /opt/mapd/omnisci-os-5.7.1-20210916-3a97ffde6c-Linux-x86_64/bin/KafkaImporter --brokers localhost:9092 --database omnisci --group-id 1 --passwd HyperInteractive --table test_kafka --topic matstream --user admin --batch 10000
Field Delimiter: ,
Line Delimiter: \n
Null String: \N
Insert Batch Size: 10000
10000 Rows Inserted, 0 rows skipped.

My config is Kafka 2.13-2.7 to create the server and the broker, using the defaults for everything, ubuntu 18.04 and Omnisci OS/Omnisci Free (tried both).

So the problem could be caused by the environment or the data; could you share the DDL of the table that’s failing and the environment you are using?

Regards,
Candido.

let me know if this will help you for further analysis.
KafkaImporterIssue_5.7.1.txt (3.1 KB)