mycontroller-org / 2mqtt Goto Github PK

MQTT Bridge

License: Apache License 2.0

Dockerfile 1.32% Go 91.98% Shell 6.70%

mqtt-bridge serial ethernet mysensors mycontroller iot home-automation bridge

2mqtt's Issues

One second buffer

It seems messages are put into a queue and held for up to 1 second. I see this in the debug logs received received received before I see about to send a message about to send a message about to send a message no matter what the rate from the serial device to 2mqtt it is holding those messages and shipping them off every 1 second.

2023-10-27T13:36:58.894+0530	debug	adapter/service.go:172	received a message from source device	{"message": "{data:{\"message\":\"metrics\",\"uptime\":46028069,\"altitude\":1072.49,\"pressure\":97675.58,\"temp\":71.66,\"heading\":83.83,\"accelX\":26.47,\"accelY\":-63.31,\"accelZ\":993.08,\"gyroX\":26.47,\"gyroY\":-63.31,\"gyroZ\":993.08}\r, others:map[], timestamp:2023-10-27 13:36:58.894284234 +0530 IST m=+2.317788558", "adapterName": "adapter1", "provider": "raw"}
2023-10-27T13:36:59.225+0530	debug	adapter/service.go:172	received a message from source device	{"message": "{data:{\"message\":\"metrics\",\"uptime\":46028402,\"altitude\":1072.62,\"pressure\":97675.14,\"temp\":71.66,\"heading\":83.98,\"accelX\":26.35,\"accelY\":-63.56,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.56,\"gyroZ\":993.32}\r, others:map[], timestamp:2023-10-27 13:36:59.224778851 +0530 IST m=+2.648283101", "adapterName": "adapter1", "provider": "raw"}
2023-10-27T13:36:59.556+0530	debug	adapter/service.go:172	received a message from source device	{"message": "{data:{\"message\":\"metrics\",\"uptime\":46028735,\"altitude\":1072.85,\"pressure\":97675.38,\"temp\":71.66,\"heading\":83.46,\"accelX\":26.35,\"accelY\":-63.68,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.68,\"gyroZ\":993.32}\r, others:map[], timestamp:2023-10-27 13:36:59.555906296 +0530 IST m=+2.979410564", "adapterName": "adapter1", "provider": "raw"}
2023-10-27T13:36:59.682+0530	debug	mqtt/device.go:134	about to send a message	{"adapterName": "adapter1", "message": "{data:{\"message\":\"metrics\",\"uptime\":46028069,\"altitude\":1072.49,\"pressure\":97675.58,\"temp\":71.66,\"heading\":83.83,\"accelX\":26.47,\"accelY\":-63.31,\"accelZ\":993.08,\"gyroX\":26.47,\"gyroY\":-63.31,\"gyroZ\":993.08}\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-10-27 13:36:58.894284234 +0530 IST m=+2.317788558"}
2023-10-27T13:36:59.682+0530	debug	mqtt/device.go:134	about to send a message	{"adapterName": "adapter1", "message": "{data:{\"message\":\"metrics\",\"uptime\":46028402,\"altitude\":1072.62,\"pressure\":97675.14,\"temp\":71.66,\"heading\":83.98,\"accelX\":26.35,\"accelY\":-63.56,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.56,\"gyroZ\":993.32}\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-10-27 13:36:59.224778851 +0530 IST m=+2.648283101"}
2023-10-27T13:36:59.683+0530	debug	mqtt/device.go:134	about to send a message	{"adapterName": "adapter1", "message": "{data:{\"message\":\"metrics\",\"uptime\":46028735,\"altitude\":1072.85,\"pressure\":97675.38,\"temp\":71.66,\"heading\":83.46,\"accelX\":26.35,\"accelY\":-63.68,\"accelZ\":993.32,\"gyroX\":26.35,\"gyroY\":-63.68,\"gyroZ\":993.32}\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-10-27 13:36:59.555906296 +0530 IST m=+2.979410564"}

I have tried setting transmit_pre_delay: 10ms and also 0s on the MQTT side, so I don't think this is where the delay is being introduced.

Is any way to disable this queueing and ship then immediately?

Thanks!

High CPU usage when using serial adapter

Hello @jkandasa,

As mentioned in the forum post https://forum.mycontroller.org/post/2990 I file here the bug about the high CPU usage when using 2mqtt as serial adapter.

I get something like this with htop:

My environment was on a Raspberry Pi with latest Raspberry Pi OS based on Debian Buster and using Mosquitto 1.5.7 from apt-get.

The config file was set like this, where /dev/serial0 is a symlink to the actual PL011 UART /dev/ttyAMA0 connected to the MySensors serial gateway node:

logger:
  mode: development
  encoding: console
  level: info

adapters:
  - name: adapter1
    enabled: true
    reconnect_delay: 20s
    provider: mysensors_v2
    source:
      type: serial
      port: /dev/serial0
      baud_rate: 115200
      transmit_pre_delay: 10ms
    mqtt:
      broker: tcp://127.0.0.1:1883
      insecure_skip_verify: false
      username:
      password:
      subscribe: mysensors-in/#
      publish: mysensors-out
      qos: 0
      transmit_pre_delay: 0s
      reconnect_delay: 5s

  - name: adapter2
    enabled: false
    reconnect_delay: 20s
    provider: mysensors_v2
    source:
      type: ethernet
      server: tcp://192.168.10.21:5003
      transmit_pre_delay: 10ms
    mqtt:
      broker: tcp://192.168.10.21:1883
      insecure_skip_verify: false
      username:
      password:
      subscribe: in_nrf24/#
      publish: out_nrf24
      qos: 0
      transmit_pre_delay: 0s
      reconnect_delay: 5s

  - name: adapter3
    enabled: false
    reconnect_delay: 20s
    provider: raw
    source:
      type: serial
      port: /dev/ttyUSB0
      baud_rate: 115200
      transmit_pre_delay: 10ms
      message_splitter: 10
    mqtt:
      broker: tcp://192.168.10.21:1883
      insecure_skip_verify: false
      username:
      password:
      subscribe: in/my_topic
      publish: out/my_topic
      qos: 0
      transmit_pre_delay: 0s
      reconnect_delay: 5s

Thanks in advance.

Documentation for serial protocol

I can't find the message format for using the bridge correctly.

I was successfully using another project but it's bridge can't receive MQTT messages from Serial (reverse way), only publish to a broker.

Here is a quick log from your application:

vd@vd1:~/downloads/2mqtt-1.3-linux-arm64$ ./2mqtt 
2023-02-17T11:53:01.168Z        info    start/logger.go:14      welcome to the 2mqtt adapter server :)
2023-02-17T11:53:01.168Z        info    start/logger.go:15      server detail   {"version": {"version":"1.3","gitCommit":"0667010e106bc840db78c3b8d157498d8e116856","buildDate":"2022-06-08T10:20:20+00:00","goLang":"go1.18.2","platform":"linux","arch":"arm64"}, "loggerConfig": {"Mode":"development","Encoding":"console","Level":"debug"}}
2023-02-17T11:53:01.169Z        debug   mqtt/device.go:60       mqtt config     {"adapterName": "adapter1", "config": {"Name":"","Broker":"tcp://127.0.0.1:1883","InsecureSkipVerify":true,"Username":"","Subscribe":"vallydroid/#","Publish":"vallydroid/","QoS":0,"TransmitPreDelay":"0s","ReconnectDelay":"5s"}}
2023-02-17T11:53:01.170Z        debug   mqtt/device.go:94       mqtt client connected successfully      {"adapterName": "adapter1", "timeTaken": "1.579829ms", "clientConfig": {"Name":"","Broker":"tcp://127.0.0.1:1883","InsecureSkipVerify":true,"Username":"","Subscribe":"vallydroid/#","Publish":"vallydroid/","QoS":0,"TransmitPreDelay":"0s","ReconnectDelay":"5s"}}
2023-02-17T11:53:01.170Z        info    adapter/service.go:238  connected to the mqtt broker    {"adapterName": "adapter1"}
2023-02-17T11:53:01.170Z        debug   mqtt/device.go:103      mqtt connection success {"adapterName": "adapter1"}
2023-02-17T11:53:01.171Z        debug   serial/device.go:58     generated config        {"config": {"Port":"/dev/ttyUSB0","BaudRate":115200,"MessageSplitter":10,"TransmitPreDelay":"10ms"}}
2023-02-17T11:53:01.171Z        info    serial/device.go:62     opening a serial port   {"adapterName": "adapter1", "port": "/dev/ttyUSB0"}
2023-02-17T11:53:01.171Z        debug   mqtt/device.go:186      subscribed a topic      {"adapterName": "adapter1", "topic": "vallydroid/#"}
2023-02-17T11:53:01.177Z        info    adapter/service.go:270  connected to the source device  {"adapterName": "adapter1"}
2023-02-17T11:53:01.177Z        info    start/start.go:26       services started        {"timeTaken": "10.175326ms"}
2023-02-17T11:53:08.511Z        debug   adapter/service.go:172  received a message from source device   {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[], timestamp:2023-02-17 11:53:08.511578315 +0000 GMT m=+7.347275376"}
2023-02-17T11:53:09.178Z        debug   mqtt/device.go:134      about to send a message {"adapterName": "adapter1", "message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-02-17 11:53:08.511578315 +0000 GMT m=+7.347275376"}
2023-02-17T11:53:09.181Z        debug   adapter/service.go:162  received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.181021406 +0000 GMT m=+8.016718560"}
2023-02-17T11:53:09.194Z        debug   adapter/service.go:162  received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.19393239 +0000 GMT m=+8.029629562"}
2023-02-17T11:53:09.992Z        debug   adapter/service.go:172  received a message from source device   {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[], timestamp:2023-02-17 11:53:09.992556235 +0000 GMT m=+8.828253407"}
2023-02-17T11:53:10.178Z        debug   adapter/service.go:148  posting a message to source device      {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.181021406 +0000 GMT m=+8.016718560"}
2023-02-17T11:53:10.178Z        debug   mqtt/device.go:134      about to send a message {"adapterName": "adapter1", "message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-02-17 11:53:09.992556235 +0000 GMT m=+8.828253407"}
2023-02-17T11:53:10.179Z        debug   adapter/service.go:162  received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.179059469 +0000 GMT m=+9.014756512"}
2023-02-17T11:53:10.186Z        debug   adapter/service.go:162  received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.186665782 +0000 GMT m=+9.022362825"}
2023-02-17T11:53:10.188Z        debug   adapter/service.go:148  posting a message to source device      {"message": "{data:[1,\"vallydroid/sensors/24AE0525BF58/send\",\"JK4FJb9YAQ0BFwEH1SoCnwDKAAAQsBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:09.19393239 +0000 GMT m=+8.029629562"}
2023-02-17T11:53:10.199Z        debug   adapter/service.go:148  posting a message to source device      {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.179059469 +0000 GMT m=+9.014756512"}
2023-02-17T11:53:10.209Z        debug   adapter/service.go:148  posting a message to source device      {"message": "{data:[1,\"vallydroid/sensors/E04066F23A08/send\",\"4EBm8joIAQ0BGgEM1SkCcwC/AAAQrhA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:10.186665782 +0000 GMT m=+9.022362825"}
2023-02-17T11:53:30.259Z        debug   adapter/service.go:172  received a message from source device   {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[], timestamp:2023-02-17 11:53:30.25971624 +0000 GMT m=+29.095413264"}
2023-02-17T11:53:31.178Z        debug   mqtt/device.go:134      about to send a message {"adapterName": "adapter1", "message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:], timestamp:2023-02-17 11:53:30.25971624 +0000 GMT m=+29.095413264"}
2023-02-17T11:53:31.178Z        debug   adapter/service.go:162  received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.178879574 +0000 GMT m=+30.014576617"}
2023-02-17T11:53:31.185Z        debug   adapter/service.go:162  received a mqtt message {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.185833108 +0000 GMT m=+30.021530151"}
2023-02-17T11:53:32.177Z        debug   adapter/service.go:148  posting a message to source device      {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.178879574 +0000 GMT m=+30.014576617"}
2023-02-17T11:53:32.188Z        debug   adapter/service.go:148  posting a message to source device      {"message": "{data:[1,\"vallydroid/sensors/184166F23A08/send\",\"GEFm8joIAQ8BHQEL1SoCiwDIAAAQuBA=\"]\r, others:map[mqtt_qos:0 mqtt_topic:vallydroid/], timestamp:2023-02-17 11:53:31.185833108 +0000 GMT m=+30.021530151"}

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.