frans-willem / aqarahub Goto Github PK

Great work! I was looking at using this project as base for a gateway replacement and to avoid extra round trips over wifi from the xiaomi gateway.

After going through many projects and clones and forks I was able to identify another project that had a bit more progress in terms of device support and documentation here: https://github.com/Koenkk/zigbee2mqtt

The guys there are really opened and there are a lot of people testing things. As you mentioned you got inspiration from zigbee-shepherd so I thought it would be great if we could also have you onboard. There are all sorts of bugs and quirks that your knowledge could help with.

Even if you don't end up joining, the project there could help you with information around adding more device support (that part was extracted in it's own library): https://github.com/Koenkk/zigbee-shepherd-converters

Another benefit of the project is that it go support for integrating into Home Assistant as an addon: https://github.com/danielwelch/hassio-zigbee2mqtt

Thanks again!

Instead of hexadecimal numbers, use the names of clusters when they are defined.

Both these devices appear to be sending something that's not a ReportAttributes command. Both are on my way from China, and I'll attempt to debug them and get them working as soon as I receive them.

hello, I have test SP-EUC01 with AquaraHub, and it can't be support.
I did some research in Aqaurahub and i found that there some key ZDO commands aren't implement, such as TC_DEV_IND、NODE_DESC_RSP、ACTIVE_EP_RSP and so on. unfortunately these commands are key to support standard zigbee device.
Mostly Aqara Device isn't standard ,but SP-EUC01.

It is possible to run this on Mac/Windows?

Reporting of attributes should be available as untyped, e.g.:

123
true
"Hello world"
[123,true,"Hello world"]

As well as typed, e.g.:

{"type":"uint8", "value": 123}
{"type":"bool", "value": true}
{"type":"octstr", "value":"Hello world"}
{"type":"struct", "value": [{"type":"uint8","value":123},{"type":"bool", "value":true},{"type":"octstr","value":"Hello world"}]}

(In the future, writing of attributes should be supported as typed. Untyped is more difficult, as we won't know what Zigbee type the attribute should be.)

Hi Frans-Willem,

I saw you have done some adding for Xiaomi devices. I was wondering if you also played around with the Aquara Cube (https://www.aliexpress.com/item/32812410838.html)? I wasn't able to get it to work on the Mozilla Gateway.

cheers
Tobias

Maybe for certain attributes force a specific attribute type or decoding method ?

I installed boost 1.60.0 manually by compiling it and installed libssl-dev alongside on my latest raspberrypi OS install.
Then I tried to compile your code and got this error

/usr/local/include/boost/asio/ssl/impl/context.ipp: In member function ‘boost::system::error_code boost::asio::ssl::context::use_rsa_private_key(const boost::asio::const_buffer&, boost::asio::ssl::context_base::file_format, boost::system::error_code&)’:
/usr/local/include/boost/asio/ssl/impl/context.ipp:716:28: error: invalid use of incomplete type ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  716 |           bio.p, 0, handle_->default_passwd_callback,
      |                            ^~
In file included from /usr/include/openssl/crypto.h:25,
                 from /usr/include/openssl/bio.h:20,
                 from /usr/include/openssl/conf.h:13,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/include/openssl/ossl_typ.h:149:16: note: forward declaration of ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  149 | typedef struct ssl_ctx_st SSL_CTX;
      |                ^~~~~~~~~~
In file included from /usr/local/include/boost/asio/ssl/context.hpp:786,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/impl/context.ipp:717:18: error: invalid use of incomplete type ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  717 |           handle_->default_passwd_callback_userdata);
      |                  ^~
In file included from /usr/include/openssl/crypto.h:25,
                 from /usr/include/openssl/bio.h:20,
                 from /usr/include/openssl/conf.h:13,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/include/openssl/ossl_typ.h:149:16: note: forward declaration of ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  149 | typedef struct ssl_ctx_st SSL_CTX;
      |                ^~~~~~~~~~
In file included from /usr/local/include/boost/asio/ssl/context.hpp:786,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/impl/context.ipp: In member function ‘boost::system::error_code boost::asio::ssl::context::do_set_password_callback(boost::asio::ssl::detail::password_callback_base*, boost::system::error_code&)’:
/usr/local/include/boost/asio/ssl/impl/context.ipp:936:14: error: invalid use of incomplete type ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  936 |   if (handle_->default_passwd_callback_userdata)
      |              ^~
In file included from /usr/include/openssl/crypto.h:25,
                 from /usr/include/openssl/bio.h:20,
                 from /usr/include/openssl/conf.h:13,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/include/openssl/ossl_typ.h:149:16: note: forward declaration of ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  149 | typedef struct ssl_ctx_st SSL_CTX;
      |                ^~~~~~~~~~
In file included from /usr/local/include/boost/asio/ssl/context.hpp:786,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/impl/context.ipp:938:16: error: invalid use of incomplete type ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  938 |         handle_->default_passwd_callback_userdata);
      |                ^~
In file included from /usr/include/openssl/crypto.h:25,
                 from /usr/include/openssl/bio.h:20,
                 from /usr/include/openssl/conf.h:13,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/include/openssl/ossl_typ.h:149:16: note: forward declaration of ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  149 | typedef struct ssl_ctx_st SSL_CTX;
      |                ^~~~~~~~~~
In file included from /usr/local/include/boost/asio/ssl/context.hpp:786,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/impl/context.ipp:940:10: error: invalid use of incomplete type ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  940 |   handle_->default_passwd_callback_userdata = callback;
      |          ^~
In file included from /usr/include/openssl/crypto.h:25,
                 from /usr/include/openssl/bio.h:20,
                 from /usr/include/openssl/conf.h:13,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/include/openssl/ossl_typ.h:149:16: note: forward declaration of ‘SSL_CTX’ {aka ‘struct ssl_ctx_st’}
  149 | typedef struct ssl_ctx_st SSL_CTX;
      |                ^~~~~~~~~~
In file included from /usr/include/openssl/engine.h:25,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:22,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/detail/impl/engine.ipp: In member function ‘const boost::system::error_code& boost::asio::ssl::detail::engine::map_error_code(boost::system::error_code&) const’:
/usr/local/include/boost/asio/ssl/detail/impl/engine.ipp:207:9: error: ‘SSL_R_SHORT_READ’ was not declared in this scope; did you mean ‘SSL_F_SSL_READ’?
  207 |         ERR_PACK(ERR_LIB_SSL, 0, SSL_R_SHORT_READ),
      |         ^~~~~~~~
In file included from /usr/local/include/boost/asio/ssl/detail/engine.hpp:163,
                 from /usr/local/include/boost/asio/ssl/detail/buffered_handshake_op.hpp:21,
                 from /usr/local/include/boost/asio/ssl/stream.hpp:29,
                 from /usr/local/include/boost/asio/ssl.hpp:24,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/detail/impl/engine.ipp:214:11: error: invalid use of incomplete type ‘SSL’ {aka ‘struct ssl_st’}
  214 |   if (ssl_->version == SSL2_VERSION)
      |           ^~
In file included from /usr/include/openssl/crypto.h:25,
                 from /usr/include/openssl/bio.h:20,
                 from /usr/include/openssl/conf.h:13,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/include/openssl/ossl_typ.h:148:16: note: forward declaration of ‘SSL’ {aka ‘struct ssl_st’}
  148 | typedef struct ssl_st SSL;
      |                ^~~~~~
In file included from /usr/include/openssl/engine.h:25,
                 from /usr/local/include/boost/asio/ssl/detail/openssl_types.hpp:22,
                 from /usr/local/include/boost/asio/ssl/context_base.hpp:19,
                 from /usr/local/include/boost/asio/ssl/context.hpp:27,
                 from /usr/local/include/boost/asio/ssl.hpp:19,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:27,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/usr/local/include/boost/asio/ssl/detail/impl/engine.ipp:221:9: error: ‘SSL_R_SHORT_READ’ was not declared in this scope; did you mean ‘SSL_F_SSL_READ’?
  221 |         ERR_PACK(ERR_LIB_SSL, 0, SSL_R_SHORT_READ),
      |         ^~~~~~~~
In file included from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/client.hpp:36,
                 from /home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt_client_cpp.hpp:8,
                 from /home/pi/AqaraHub/src/mqtt_wrapper_impl.h:3,
                 from /home/pi/AqaraHub/src/mqtt_wrapper.cpp:4:
/home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/endpoint.hpp: In member function ‘bool mqtt::endpoint<Socket, Mutex, LockGuard>::handle_close_or_error(const boost::system::error_code&)’:
/home/pi/AqaraHub/submodules/mqtt_cpp/include/mqtt/endpoint.hpp:2355:61: error: ‘boost::asio::ssl::error’ has not been declared
 2355 |             ERR_GET_REASON(ec.value()) == boost::asio::ssl::error::stream_truncated
      |                                                             ^~~~~
make[2]: *** [CMakeFiles/common.dir/build.make:173: CMakeFiles/common.dir/src/mqtt_wrapper.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:126: CMakeFiles/common.dir/all] Error 2
make: *** [Makefile:149: all] Error 2

Which dependency version do I need?

When linking the WXKG02LM (dual-button wall-switch), one of the attributes can not be decoded.

<DEBG> [FRAME] << AREQ AF_INCOMING_MSG 00 00 00 00 51 17 01 01 00 83 00 a0 a7 00 00 00 23 1c 5f 11 01 0a 01 ff 42 1b 01 21 bd 0b 03 28 1c 04 21 a8 01 05 21 41 01 06 24 01 00 00 00 00 0a 21 00 00 51 17 1d
<DEBG> [MSG] SrcAddr: 5969, ClusterId: 0, SrcEndpoint: 1
<DEBG> [MSG] Payload: 01 ff 42 1b 01 21 bd 0b 03 28 1c 04 21 a8 01 05 21 41 01 06 24 01 00 00 00 00 0a 21 00 00
<DEBG> [FRAME] >> SREQ UTIL_ADDRMGR_NWK_ADDR_LOOKUP 51 17
<DEBG> [FRAME] << SRSP UTIL_ADDRMGR_NWK_ADDR_LOOKUP 19 f5 83 01 00 8d 15 00
DEcoding type string 66
<DEBG> [MSG] Exception while handling message:Not enough data to decode integer

It appears the string (0x42) with length 0x1B, is only 0x1A long. Not sure how to handle this. Possibly allow for early-end in string decoding ?

Current properties are copied as-is from zigbee-shepherd's debug output.
I think at least the PAN ID and EPAN ID should be somewhat more random, possibly using the coordinators EID.
I'm not sure if the Xiaomi devices support other pre-shared network keys.

Hello sir, I tested several devices with AqaraHub, such as smar plug (SP-EUC01), Rubik's cube controller and mini switch. The Rubik's cube controller and mini switch can join normally, but after joining the network with smart plug, it will cause a few exceptions:

1. After the smart plug (SP-EUC01) is added, the original network access time is 60s, but it will become longer. After about 3-4 minutes, the log will show that the network access time is over.
2. During this period (3-4 minutes after the smart plug is connected to the network), other devices can access the network, but the way to access the network seems to be similar to the smart plug (SP-EUC01).
3. After 3-4 minutes, the log prints the end of the network connection time, and then the network is turned on again, but no device can be connected. The program can run normally only after restarting.

The above phenomenon was obtained after I carefully and repeatedly verified it. Therefore, I boldly guessed that this problem was caused by the smart plug (SP-EUC01) joining the network due to different processing methods.
I added the log of my test to the attachment, including a series of procedures for the device to go online as described above, and a log of all the information that the mqtt subscribes to when the program is running. Among them, I made step instructions.
log.zip

Originally posted by @DuanYuXian in #59 (comment)

when I copy AqaraHub to other folder it can not run
root@mdmserver:/tmp# ./AqaraHub -p /dev/ttyACM0
[Main] Serial port: /dev/ttyACM0
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injectorboost::property_tree::info_parser::info_parser_error >'
what(): ../clusters.info: cannot open file for reading

Hi,

I wanted to install the service so my Hub automatically starts when I boot the PC where I have it installed.
So I looked at the AqaraHub.service file and saw this line:

EnvironmentFile=/etc/default/AqaraHub

What vars can I set in that file?
Can you please provide a sample file?

Thank you :)

See https://github.com/Frans-Willem/AqaraHub/blob/master/documentation/devices/lumi.sensor_smoke.md

"MQTT authentication with username/password" - Is this possible?

Hunter package manager can be used instead of conan. Hunter is a CMake driven cross-platform package manager for C/C++. And it doesn't require any third-party software for dependency managment (it uses CMake scripts for dependency managment).

This update will simplify the "Getting started" guide for developers, because all dependencies (including Boost libraries) will be installed automatically after opening project in IDE or running cmake from command line.

Turning on/off the Xiaomi Smart Plug ZNCZ02LM requires sending commands, e.g. to cluster 0x0006, endpoint 0x01, command 0x00 (off), 0x01 (on), or 0x02 (toggle).

Some commands require a payload, so a possible solution would be to send
[] to root/command/158D00015D231D/1/OnOff/On
(extended id, endpoint, cluster, command).

Payload could be implemented by giving a list of JSON-encoded ZclVariants, possibly dropping the header...

Continuing on issue #24:

About the choice between of plain values instead of JSON, I've considered it in the past, but decided against it for symmetry. e.g. when receiving a value from Zigbee, it's obviously easy to just serialize it as the plain value, but then when trying to send something from MQTT to Zigbee 1 could be interpreted as a uint8, int8, uint16, uint24, ..., single, float, double, and there's really no way to know which one the zigbee device will support.

It is ok for me to use JSON for sending commands, as it could be "constructed" as simple strings, without any JSON support. But for reading received values I have to use some JSON parser.

I know that OpenHAB supports javascript (and thus JSON) transforms, and Domoticz needs some kind of translation like Node-Red inbetween anyway that can tackle the JSON stuff, which is why I figured it wouldn't be a very big problem. May I ask what kind of home automation setup you'd like to run that wouldn't support JSON?

Now, the primary master of my home automation is PLC with IEC61131-3 programming (Foxtrot from Teco), where I have programmed a MQTT client. Parsing JSON is possible there, but it adds a unnecessary overhead because it is limited, specially in string resources.

Next I use some Arduino/ESP based gadgets, all connected to MQTT broker. And there I found it as a big overhead.

Hello
I've read https://github.com/Frans-Willem/AqaraHub/blob/master/documentation/mqtt-topics.md but still can't control zigbee devices
for example i've sent to topic AqaraHub/00158D0001A35845/1/out/OnOff/OnOff value 0 or 1
please share the logic for topic and value forming?
thank you

This issue is related to #40.
we have port it to openwrt , however there is a new issue occurs: it trigger segment fault error. The error.log is following ,pls help.
err.log

In some cases, when multiple messages are still in the serial buffer when starting up, the request/response buffer gets out of sync.
I suspect that when receiving a RESET_IND, I should mark all requests in the queue as failed.

Currently PermitJoin is sent just after initializing everything.
It'd be better if you could send something on MQTT to trigger this behaviour manually.

Sorry for the question, but how can I integrate it with domoticz? The device works, I paired some devices:
00 20 59 c8 16 1d
[Report] Attempting to look up full source address for 5832
[FRAME] >> SREQ UTIL_ADDRMGR_NWK_ADDR_LOOKUP c8 16
[FRAME] >> SREQ UTIL_ADDRMGR_NWK_ADDR_LOOKUP c8 16
[FRAME] << SRSP UTIL_ADDRMGR_NWK_ADDR_LOOKUP 60 4f 9f 01 00 8d 15 00
[Report] Publishing to domoticz/in/report/00158D00019F4F60/1/Basic/0xFF02: {"type":"struct","value":[{"type":"bool","value":false},{"type":"uint16","value":3025},{"type":"uint16","value":5032},{"type":"uint40","value":1},{"type":"uint16","value":101},{"type":"uint8","value":89}]}
[Report] All done
Domoticz I created a mqtt client gateway with lan interface. What's next? How do you find the paired tools in domoticz?

When AqaraHub application is started before MQTT server, the connection fails and no messages are delivered even there are messages like this one in log:

Jun 19 16:56:22 mqtt2 env[1981]: [Report] Publishing to AqaraHub/report/00158D0002132F39/1/OnOff/OnOff: {"type":"bool","value":false}

Also when started correctly and MQTT server is restarted, AqaraHub never reconnects again.

Smoke sensor appears to report Zone Status Change notification, we should handle and parse this properly. From what I could decode manually:

Zone Status: 0x0001 for no smoke, 0x0000 for smoke
Extended Status: 0x00
Zone ID: 0xFF
Delay: 0x0000

Currently commands always send empty payload. Ideally I'd like to implement something to the effect of:

[{"type":"uint8","value":12}, {"type":"uint16", "value":1234}]

Where each item in the array will be converted to a ZclVariant, but will be encoded on the line without the type.

Thanks for the great project. It works for me in general and I think it is much better than all other JS crap.
But I am facing one problem at daemon start. After reboot or ZigBee stick plug-in I always have to start the daemon two times, as the first start is not able to reset the device. I have to kill the process and on second try it is initialized properly.

This is log of the first start:

May 31 20:11:01 mqtt2 env[707]: [Main] Serial port: /dev/ttyACM0
May 31 20:11:01 mqtt2 env[707]: [Main] Setting up ZNP connection
May 31 20:11:01 mqtt2 env[707]: [Main] Setting up MQTT connection
May 31 20:11:01 mqtt2 env[707]: [MqttWrapper] Creating connection to localhost : 1883
May 31 20:11:01 mqtt2 env[707]: [Main] Using MQTT prefix 'AqaraHub/'
May 31 20:11:01 mqtt2 env[707]: [Main] Using PSK xx xx xx xx xx xx xx xx 00 00 00 00 00 00 00 00
May 31 20:11:01 mqtt2 env[707]: IO Service starting
May 31 20:11:01 mqtt2 env[707]: [Initialize] Doing initial reset, without clearing config or state
May 31 20:11:01 mqtt2 env[707]: [FRAME] >> SREQ SAPI_WRITE_CONFIGURATION 03 01 00
May 31 20:11:01 mqtt2 env[707]: [MqttWrapper] ConnAckHandler: clean=false
.....
May 31 20:11:59 mqtt2 env[707]: [FRAME] << AREQ SYS_RESET_IND 00 02 00 02 06 03

Since switching to boost coroutines, it's possible to compile without conan, often with the system compiler, and likely even with the system boost libraries. I should rework the readme to reflect this.

Furthermore the Readme should have a summary of the current state, how useful it is, and what it will and won't do at this stage.

...

Hi Frans,
I found a weird issue about the ZED joining with ZR through the smart plug (SP-EUC01) which plays a role as ZR.
I have an environment consists of a CC2530 flashed https://github.com/Koenkk/Z-Stack-firmware/blob/master/coordinator/Z-Stack_Home_1.2/bin/default/CC2530_DEFAULT_20190608.zip, a smart plug and a ZED. The testing steps are bellow:
(1) Open the network of the CC2530.
(2) Reset the smart plug and let it join the CC2530 successfully.
(3) Reset a ZED and let it join the CC2530 through the smart plug.
(4) After 180 seconds, open the network of the CC2530, repeat the step 3, and the ZED can't join CC2530 through the smart plug.

I turned on a sniffer and catched some packets. https://github.com/4djason/Z-Stack-firmware/tree/master/log , the log file is multi-hop-joining.pcapng, you can follow the README to view the encrypted part.
It starts at No.7901, the smart plug try to send Update Device Request to the CC2530, but the CC2530 send Remove Device Request to the smart plug to kick out the ZED.
Both network of the ZC and ZR were opened and associations were permitted, You can see them at No.7884 and No.7885.
I can't figure out what is going wrong, maybe there are some bugs of trust center in CC2530?

I will be appreciate if you can help me, thks.

Hi,

I already tried your project with a single sensor and liked it.

Now I want to install all my sensors.
How can I backup the configured sensors configuration so I don't need to pair them all again when my node I run AqaraHub on would break?

Dear,
I've followed your instructions & was able to build & run OK the binary in an Ubuntu 20.04 64 bits (please see log @the end)
I've also flashed the dongle exactly with the same version you used:
https://github.com/Koenkk/Z-Stack-firmware/raw/b2a0be7c4c84c2acf6ecd292f7afd0f488b3b9b1/coordinator/CC2531/bin/CC2531ZNP-Prod_20190223.zip

But i'm not able to put it in discover mode in order to join devices, when I send (i have a mosquitto server up & running @localhost):
mosquitto_pub -h localhost -t AqaraHub/write/permitjoin -m 60
I never get the "AqaraHub/report/permitjoin topic, with the same number" response that you mention.
Any hint/advice/clue will be highly appreciated.Thanks in advance!

###LOG###
~/wk/AqaraHub/build$ sudo ./AqaraHub --port /dev/ttyACM0 --mqtt mqtt://localhost/ --topic AqaraHub
[Main] Serial port: /dev/ttyACM0
[Main] Setting up ZNP connection
[Main] Setting up MQTT connection
[MqttWrapper] Params: { Unencrypted, TCP, localhost, 1883, --, (no password), AqaraHub }
[MqttWrapper] Connecting...
[Main] Using MQTT prefix 'AqaraHub/'
[Main] Recursively publishing object and array properties
[Main] No PSK specified on command-line, using default ('AqaraHub')
[Main] Using PSK 41 71 61 72 61 48 75 62 00 00 00 00 00 00 00 00
IO Service starting
[Initialize] Doing initial reset (this may take up to a full minute after a dongle power-cycle)
[FRAME] >> AREQ SYS_RESET 01
[MqttWrapper] Connected, clean=false
[FRAME] << AREQ SYS_RESET_IND 00 02 00 02 06 03
[Initialize] Building desired configuration
[FRAME] >> SREQ SAPI_GET_DEVICE_INFO 01
[FRAME] << SRSP SAPI_GET_DEVICE_INFO 01 84 70 38 19 00 4b 12 00
[Initialize] Device IEEE Address: 124b0019387084
[Initialize] Verifying full configuration
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 8f
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 63
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 62
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 87
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 84
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 2d
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 83
[FRAME] >> SREQ SAPI_READ_CONFIGURATION 03
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 8f 01 01
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 63 01 00
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 62 10 41 71 61 72 61 48 75 62 00 00 00 00 00 00 00 00
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 87 01 00
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 84 04 00 08 00 00
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 2d 08 84 70 38 19 00 4b 12 00
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 83 02 ff ff
[FRAME] << SRSP SAPI_READ_CONFIGURATION 00 03 01 00
[Initialize] Desired configuration matches current configuration, ready to start!
[Initialize] Starting ZDO
[FRAME] >> SREQ SAPI_GET_DEVICE_INFO 00
[FRAME] >> SREQ ZDO_STARTUP_FROM_APP 64 00
[FRAME] << SRSP SAPI_GET_DEVICE_INFO 00 00 01 00 64 07 00 0e 01
[WaitForState] Subscribing to on_state_change_ event
[FRAME] << SRSP ZDO_STARTUP_FROM_APP 00
[Initialize] ZDO Start return value: 0
[FRAME] << AREQ ZDO_STATE_CHANGE_IND 09
[WaitForState] Got on_state_change_
[Initialize] Final device state 9
[FRAME] >> SREQ ZDO_MGMT_PERMIT_JOIN_REQ 02 00 00 00 00
[FRAME] << SRSP ZDO_MGMT_PERMIT_JOIN_REQ 00
[FRAME] << AREQ ZDO_PERMIT_JOIN_IND 00
[FRAME] << AREQ ZDO_MGMT_PERMIT_JOIN_RSP 00 00 00
[FRAME] >> SREQ AF_REGISTER 01 04 01 05 00 00 00 00 00
[FRAME] << SRSP AF_REGISTER 00
[MqttWrapper] Sending async subscribe directly
[Main] Initialization complete!
[Main] In final handler

I know, it's AqaraHub not HueHub, but I was hoping to avoid moving away from AqaraHub.

<DEBG> [OnPermitJoin] Published OK
<DEBG> [FRAME] << AREQ ZDO_MGMT_PERMIT_JOIN_RSP 00 00 00
<DEBG> [PermitJoin] Permit join OK
<DEBG> [FRAME] << AREQ ZDO_TC_DEV_IND 70 37 9c f6 63 04 01 88 17 00 00 00
<DEBG> [ZnpApi] Unhandled frame AREQ ZDO_TC_DEV_IND
<DEBG> [FRAME] << AREQ ZDO_END_DEVICE_ANNCE_IND 70 37 70 37 9c f6 63 04 01 88 17 00 8e
<DEBG> [ZnpApi] Unhandled frame AREQ ZDO_END_DEVICE_ANNCE_IND

I had a look at #59 but couldn't work out what my device address was.

For now we're calling UTIL_ADDRMGR_NWK_ADDR_LOOKUP and UTIL_ADDRMGR_EXT_ADDR_LOOKUP whenever we need to convert between the two, but that's sending a lot of data over the serial line.
Ideally we should cache this and not call the USB stick all the time.

For battery operated type of sensors the battery seems reported in Voltage value in the beginning of 0xFF01 xiaomi specific data stream like : index: 1, type: uint16, value: (C7 0B) = 3015 in milliVolts (3.015Volt)
Suggested range for the percentage calculation is 2.4V .. 3V

It'd be nice to have clusterdb.info compiled in and then only overridden if needed.

When a proper name is available for an attribute, use it in reporting.

After hitting ctrl-C and restarting AqaraHub, I get the following:

voltagex@panopticon:~/output-known/aqara $ sudo ./AqaraHub -p /dev/ttyACM0
<INFO> [Main] Serial port: /dev/ttyACM0
<INFO> [Main] Setting up ZNP connection
<INFO> [Main] Setting up MQTT connection
<INFO> [MqttWrapper] Connecting...
<INFO> [Main] Using MQTT prefix 'AqaraHub/'
<INFO> [Main] Recursively publishing object and array properties
<INFO> [Main] No PSK specified on command-line, using default ('AqaraHub')
<INFO> [Main] Using PSK 41 71 61 72 61 48 75 62 00 00 00 00 00 00 00 00
IO Service starting
<DEBG> [Initialize] Doing initial reset (this may take up to a full minute after a dongle power-cycle)
<DEBG> [FRAME] >> AREQ SYS_RESET 01
<DEBG> [MqttWrapper] Connected, clean=false
<DEBG> [FRAME] << AREQ SYS_RESET_IND 00 02 00 02 06 03
<DEBG> [Initialize] Building desired configuration
<DEBG> [FRAME] >> SREQ SAPI_GET_DEVICE_INFO 01
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION
<DEBG> [FRAME] << SRSP SYS_VERSION 02 00 02 06 03
<DEBG> [ZnpApi] Unhandled frame SRSP SYS_VERSION

(output repeats)

with --recursive-publish option hub also sends full json string. but my mqtt client module independently parses the json payloads, and there is duplication. since client does it badly, I want the hub to do it, but did not send full json. is it possible?

LinkQuality value reported in znp(?) level messages and related to wireless coverage of end devices. Bringing it up to mqtt level it could be vital to check the health of the network.

Maybe somebody port this project to openwrt

I think I'm making multiple mistakes here, but I've got the following cmake file to build for the (OpenWRT) GL.inet AR750S router.

SET(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
SET(BUILD_SHARED_LIBS OFF)
SET(CMAKE_EXE_LINKER_FLAGS "-static")

set (CMAKE_SYSTEM_NAME Linux)
set (CMAKE_C_COMPILER mips-openwrt-linux-musl-gcc)
set (CMAKE_CXX_COMPILER mips-openwrt-linux-musl-g++)

set (THREADS_PREFER_PTHREAD_FLAG On) #https://gitlab.kitware.com/cmake/cmake/issues/16540
set (BOOST_INCLUDEDIR /home/voltagex/src/openwrt/staging_dir/target-mips_24kc_musl/usr/)
set (BOOST_LIBRARYDIR /home/voltagex/src/openwrt/staging_dir/target-mips_24kc_musl/usr/)
set (BOOST_ROOT /home/voltagex/src/openwrt/staging_dir/target-mips_24kc_musl/usr/)
set (OPENSSL_INCLUDE_DIR  /home/voltagex/src/openwrt/build_dir/target-mips_24kc_musl/openssl-1.1.1g/include/)
set (OPENSSL_CRYPTO_LIBRARY  /home/voltagex/src/openwrt/build_dir/target-mips_24kc_musl/openssl-1.1.1g/)

One minor tweak of CMakeLists is needed as OpenWRT doesn't keep old boost versions around

diff --git a/CMakeLists.txt b/CMakeLists.txt
index fb844a5..770af55 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -15,7 +15,7 @@ set(CMAKE_CXX_STANDARD 14)
 find_package(Threads REQUIRED)

 set(Boost_USE_STATIC_LIBS ON)
-find_package(Boost 1.60.0 COMPONENTS system program_options log coroutine unit_test_framework REQUIRED)
+find_package(Boost 1.73.0 COMPONENTS system program_options log coroutine unit_test_framework REQUIRED)

 # STLab library
 find_path(STLAB_INCLUDE_DIRS NAMES stlab/version.hpp PATHS ${CMAKE_SOURCE_DIR}/submodules/stlab-libraries/)

I'm trying to build statically as the manufacturer of the AR750S has an odd setup that I can't replicate exactly

With the toolchain, boost and OpenSSL from OpenWRT buildroot, I can get most of the way through the build, but for some reason it tries to pull in a host SSL library instead of the one I've pointed it to: /home/voltagex/src/openwrt/staging_dir/toolchain-mips_24kc_gcc-8.4.0_musl/lib/gcc/mips-openwrt-linux-musl/8.4.0/../../../../mips-openwrt-linux-musl/bin/ld: attempted static link of dynamic object /usr/lib/x86_64-linux-gnu/libssl.so'`

By some reports compiling on a raspberry pi takes almost an hour, and requires a 1G swap file.
We should look into cross compilation. Targetting Raspbian for RPi 1 and 2/3 would probably be helpful to a lot of users.