casteer / mqtt-data-logger Goto Github PK

Simple Python MQTT Data Logger

This software uses the Python logger to create a logfile for all messages for all topics to which this MQTT client has subscribed. Note: by default it will only log changed messages. This is for sensors that send out their state a regular intervals bu that state doesn't change The program is run from the command line You can subscribe to multiple topics.

You need to provide the script with:

List of topics to monitor
broker name and port
username and password if needed.
base log directory and number of logs have defaults

Valid command line Options: --help -h -b -p -t -q -v -d logging debug -n -u Username -P Password -s
-l -r
-f <number of log files default= unlimited"

Example Usage:

You will always need to specify the broker name or IP address and the topics to log

Note: you may not need to use the python prefix or may need to use python3 mqtt_data_logger.py (Linux)

Specify broker and topics

python mqtt_data_logger.py -b 192.168.1.157 -t sensors/#

Specify broker and multiple topics

python mqtt_data_logger.py -b 192.168.1.157 -t sensors/# -t  home/#

Log All Data:

python mqtt_data_logger.py b 192.168.1.157 -t sensors/# -s 

Specify the client name used by the logger

python mqtt_data_logger.py b 192.168.1.157 -t sensors/# -n data-logger

Specify the log directory

python mqtt_data_logger.py b 192.168.1.157 -t sensors/# -l mylogs

Logger Class

The class is implemented in a module called m_logger.py (message logger).

To create an instance you need to supply three parameters:

The log directory- defaults to mlogs
Number of records to log per log- defaults to 1000
Number of logs. 0 for no limit.- defaults to 0

log=m_logger(log_dir="logs",log_recs=1000,number_logs=0):

The logger creates the log files in the directory using the current date and time for the directory names.

The format is month-day-hour-minute e.g.

You can log data either in plain text format or JSON format.

To log data either in plain text then use the

log_data(data) method.

To log data as JSON encoded data call the

json_log(data) method.

Both method takes a single parameter containing the data to log as a string, list or dictionary..

e.g.

log.log_data(data) or log.log_json(data)

#The log file will contain the data as #plain text or JSON encoded data strings #each on a newline.

The logger will return True if successful and False if not.

To prevent loss of data in the case of computer failure the logs are continuously flushed to disk .

Read more about this application here: http://www.steves-internet-guide.com/simple-python-mqtt-data-logger/