High Level Node.js Kafka Client

The most advanced Kafka Client.

Features

• easy promise based API
• a lot of Kafka pitfalls already taken care of
• backpressure and stream consume modes
• secure committing in backpressure (1:n, batch) mode
• plain Javascript implementation based on kafka-node and a super fast native implementation based on node-rdkafka
• SSL, SASL & Kerberos support
• auto reconnects
• auto partition recognition and deterministic spreading for producers
• intelligent health-checks and analytic events for consumers and producers

You might also like

• check out :goberserk: node-kafka-streams for a stream processing kafka api
• check out 🔥 node-kafka-connect for a easy datastore <-> kafka transfer

Latest Changes

Can be found here

Install

npm install --save sinek

Usage

Usage - JS Client (based on kafka.js)

const {
  JSConsumer,
  JSProducer 
} = require("sinek");
 
const jsProducerConfig = {
  clientId: "my-app",
  brokers: ["kafka1:9092"]
}
 
(async () => {
 
  const topic = "my-topic";
 
  const producer = new JSProducer(jsProducerConfig);
  const consumer = new JSConsumer(topic, jsConsumerConfig);
 
  producer.on("error", error => console.error(error));
  consumer.on("error", error => console.error(error));
 
  await consumer.connect();
 
  // consume from a topic.
  consumer.consume(async (messages) => {
    messages.forEach((message) => {
      console.log(message);
    })
  });
 
  // Produce messages to a topic.
  await producer.connect();
  producer.send(topic, "a message")
})().catch(console.error);
 

Further Docs

• Best-practice example
• SSL example
• SASL+SSL example
• Alpine based docker example
• Debian based docker example

make it about them, not about you

• Simon Sinek