Code Monkey home page Code Monkey logo

simple-kafka-mqtt-connector's Introduction

A simple MQTT to Apache Kafka Connector - ARM64 Edition

This is a fork of the original that is specifically geared toward ARM64 compatibility.

Usage

Prerequisites

Java 17 or higher

Note that you should be able to use Java 8 or 11 with no problem, but the Docker container is built with 17 and all testing/development has been done with 17.

Configuration

  • Edit the application.properties file

  • kafka.host = [HOSTNAME|IP] - IP THROUGH WHICH THE KAFKA BROKER WILL BE REACHED

  • kafka.port = [INTEGER]

  • kafka.client.id = [STRING]

  • mqtt.host = [HOSTNAME|IP] - IP THROUGH WHICH THE MQTT BROKER WILL BE REACHED

  • mqtt.port = [INTEGER]

  • mqtt.client.id = [STRING]

  • mqtt.qos = [INTEGER] - Quality of service for MQTT - Allowed[0,1,2]

  • topic.mapping = [list_of_topic_mapping] - This is a semicolon-separated list that describes how topics are routed from MQTT to Kafka - (Separators >>> (between kafka/MQTT mappings) and ; (between pairs))

    • Example: mqttTopicA>>>kafkaTopicA;mqttTopicB>>>kafkaTopicB;mqttTopicC>>>kafkaTopicC

More info about wildcards in MQTT topics can be found here: HiveMQ's MQTT Topics & Best Practices (pt 5)

How to build

  • Run: mvn clean install

How to run

  • Place the jar with dependencies and your edited application.properties-file in the same directory
  • Open a bash or CMD in the same directory as the .jar
  • Run: java -jar kafka-mqtt-connector-0.1.0-SNAPSHOT-jar-with-dependencies.jar

Usage with Docker

Most basic Docker build command:

docker build -t mikeoertli/kafka-mqtt-connector:0.1.0 -t mikeoertli/kafka-mqtt-connector:latest .
docker run -d -t -i \
  -e KAFKA_HOST='localhost' \
  -e KAFKA_PORT=9092 \
  -e KAFKA_CLIENT_ID='testing-kafka-producer-1' \
  -e MQTT_HOST='localhost' \
  -e MQTT_PORT=1883 \
  -e MQTT_CLIENT_ID='mqtt-client-1' \
  -e MQTT_QOS=2 \
  -e TOPIC_MAPPING='iot/event/#>>>iot_event;iot/status/#>>>iot_status;iot/tasking/#>>>iot_tasking'  \
  --name kafka-mqtt-connector \
  mikeoertli/kafka-mqtt-connector:latest

Usage with Docker-Compose

Standalone

Full Stack

  • Full Stack (mqtt-broker (via mosquitto), zookeeper, kafka-broker, kafka-mqtt-connector)
  • docker-compose.yml
  • env.list
  • Place docker-compose.yml and env.list in the same directory
  • Edit env.list to your needs!
  • Run: docker-compose up -d

License

See LICENSE file for License

simple-kafka-mqtt-connector's People

Contributors

agrg-ipa-070 avatar arthurgrigo avatar fossabot avatar mikeoertli avatar snyk-bot avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.