Code Monkey home page Code Monkey logo

node-red-contrib-kafka-client's Introduction

@hylink/node-red-kafka-client

This node can be used in order to produce and consume messages to Kafka. It is highly depends on 'kafka-node' library. Consists three nodes.

  • kafka-broker
  • kafka-producer
  • karka-consumer

Note: This library add sasl/plain supports base on node-red-contrib-kafka-client project

Input Parameters

kafka-broker

Name (Optional)

Name wanted to be shown in Node

Hosts

Host names comma delimited (Multiple host is provided)

Use TLS

Check if tls security is required for Kafka Cluster

CA Certs (Optional)

CA Root certificate path defined in Kafka Cluster

Client Cert (Optional)

Client cert path created by openssl derived from Private Key (pem)

Private Key (Optional)

Private Key path created by openssl (pem)

Passphare (Optional)

Passphrase of created private Key

Self Sign

Check if want to be allowed untrusted certificates

Use SASL

Check if SASL security is required for Kafka Cluster

SASL Username (Optional)

SASL Username

SASL Password (Optional)

SASL Password

kafka-producer

Name (Optional)

Name wanted to be shown in NodeRename your node

Broker

Broker which is wanted to be connect

Topic

Topic name of selected broker which is wanted to be consume

Require Ack

Default value is 1. 0 can be past if Acknowledge is not required.

Ack Timeout

Timeout of acknowledge response.

Attiributes

Can be selected if compression is important

kafka-consumer

Name (Optional)

Name wanted to be shown in NodeRename your node

Broker

Broker which is wanted to be connect

Topic

Topic name of selected broker which is wanted to be consume

From Offset

'latest', 'none' or 'earliest' options can be selected

Out of Range Offset

'latest', 'none' or 'earliest' options can be selected

Installation

npm install node-red-contrib-kafka-client

Usage

  1. Example JSON here
{"topic":"TOPIC_NAME","value":"DENEME","offset":16638,"partition":0,"highWaterOffset":16639,"key":null,"timestamp":"2020-08-19T08:58:27.866Z"}

Screenshots

kafka-broker

kafka-consumer

kafka-flow

kafka-producer

node-red-contrib-kafka-client's People

Contributors

emrebekar avatar carldolling avatar jpwsutton avatar rparaujo avatar

Stargazers

专业小白 avatar

node-red-contrib-kafka-client's Issues

When broker becomes not available, the producer nodes make NodeRED crashing

Node-RED version: v3.1.9, Node.js version: v16.20.2 (Node-RED docker image nodered/node-red).

If the producer node is connected to a broker and the broker becomes unavailable, the node raise an uncaught exception and Node-RED crashes.

17 May 08:09:35 - [error] [producer:e4bd81b644e8a4b1] NestedError: refreshBrokerMetadata failed
17 May 08:09:45 - [red] Uncaught Exception:
17 May 08:09:45 - [error] NestedError: refreshBrokerMetadata failed
    at /data/node_modules/kafka-node/lib/kafkaClient.js:378:35
    at /data/node_modules/async/dist/async.js:473:16
    at next (/data/node_modules/async/dist/async.js:5332:29)
    at /data/node_modules/async/dist/async.js:972:16
    at wrappedFn (/data/node_modules/kafka-node/lib/kafkaClient.js:481:14)
    at /data/node_modules/kafka-node/lib/kafkaClient.js:860:7
    at Map.forEach (<anonymous>)
    at KafkaClient.clearCallbackQueue (/data/node_modules/kafka-node/lib/kafkaClient.js:857:9)
    at Socket.<anonymous> (/data/node_modules/kafka-node/lib/kafkaClient.js:787:12)
    at Socket.emit (node:events:513:28)
Caused By: BrokerNotAvailableError: Broker not available (socket closed)
    at new BrokerNotAvailableError (/data/node_modules/kafka-node/lib/errors/BrokerNotAvailableError.js:11:9)
    at Socket.<anonymous> (/data/node_modules/kafka-node/lib/kafkaClient.js:778:19)
    at Socket.emit (node:events:513:28)
    at TCP.<anonymous> (node:net:301:12)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.