mblackstock / node-red-contrib-influxdb Goto Github PK

Within a single Measurement I would like to assign different Tags to Fields and hopefully also define different retention policies for individual Fields. Is this something that can be done within a single Batch update function or is it something where I have to have functions for each unique combination and then have them use the same timestamp for each to help keep the database optimized?

Hi, I'm trying to insert a value in InfluxDb with just a timestamp and tags:

This works fine when setting a field, here "value":
msg.payload = [
{value: 1, time: 1},
{tag: "Test"}
];

This doesn't work with just the timestamp (no "value" field):
msg.payload = [{time: 1}, {tag: "Test"}];

=> Error: A 400 Bad Request error occurred:
{"error":"unable to parse 'mymeasurement,tag=Test': invalid field format"}

Same exemple works using the CLI:

insert mymeasurement tag="Test" 1
select * from mymeasurement
name: mymeasurement
time tag
1 Test

Workaround is to add a dummy field, but ideally I'd like not to

I am using Influx Cloud 2.0. In this hosted influxdb, the access is regulated by a token auth, but I don't see how to use your node to connect to influx cloud.

I had a working flow with influx v0.13 and earlier node-red-contrib-influxdb.

I installed a new RPI3 with the actual sw components (Influx 1.2.2-1) and the same flow and all the influxdb output nodes in node-red produce the following error at deploying:

TypeError: Object function Object() { [native code] } has no method 'assign'

The function before influxdb node as an example is the following: ( I wouldn't think that the problem is here)

msg.payload = [{
value: Number(msg.payload)
},
{
sensor:"moisture"
}];
return msg;

Looks like a problem with the oneditprepare code in that node's html file.

Hello,

It should be good to add an output to "influxdb out" node as callback, for example for me to notify success on the node red dashboard and empty the form.

What do you think ?

Regards,

28 May 12:03:15 - [info] Node-RED version: v1.0.6
28 May 12:03:15 - [info] Node.js version: v12.13.0
28 May 12:03:15 - [info] Darwin 18.7.0 x64 LE

While the 'influxdb out' node is working fine, the 'influx batch' node does not work.

Using the example flow provided in the nodes flow page, the following error is reported in a catch node:

Error: write EPROTO 4488598976:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:

and the stack shows

Error: write EPROTO 4488598976:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332: at WriteWrap.onWriteComplete [as oncomplete] (internal/stream_base_commons.js:87:16)

when I inset one record in an influxdb database I get the record inserted twice with a very little difference in time:
Record format (passed as msg.payload):

If I query the database

> Select * FROM meteo WHERE time >= '2019-03-20T08:05:00Z' and "deviceId"='Jardin' limit 2
name: meteo
time                           HPa  deviceId hAire hCrudo hSuelo l/m2 temp
----                           ---  -------- ----- ------ ------ ---- ----
2019-03-20T09:09:13.709229151Z 1030 Jardin   56    195    36     0    7.1
2019-03-20T09:09:13.712843732Z 1030 Jardin   56    195    36     0    7.1
>

I only call influxdbout once, but there are two records with less than 4/1000 of a second difference.
Is this a known bug or I am doing something wrong? (I have not found it reported)
Regards

Trying to utilize influxdb and tensorflow to train from large datasets. Would like to 'stream' the query, transform it to tensorflow dataset to be consumed by tensorflow js. If this is out of scope of this project, please let me know.

Suggested by Julian - https://groups.google.com/forum/#!searchin/node-red/influxdb%7Csort:relevance/node-red/BJp_MEhoZIE/QbfSZwbaCwAJ

Hi, thank you for your node-plugin!

I've not been able to test it yet as I'm using the 2.0 Cloud version of InfluxDB.
Please excuse my usage of the "Issue-Tracker" as I don't know how to start a normal conversation over here.

May it be possible to support the InfluxDB 2.0 Cloud (API) to write (and read) to the InfluxDB Cloud database?
I would like to run InfluxDB in the cloud as my Node Red instance runs on an internet server, too. Bus therefore I cannot run InfluxDB on a local machine.

I would be very grateful.
Thank you! Best regards!

Port is set to 8086

It seems to be making a request to port 80

InfluxDB works:

I today detected an issue when using the "out" node.
We tried to use the provided sample but the inserted sets were always dated to UNIX start date in 1970. After some time we recognized that everything works fine if we do not use the ".getTime()" for the Date object.

I used:
InfluxDB: 1.7.10
node-red-contrib-influxdb 0.4.0
Node-RED 1.0.4 on latest non-LTS NodeJS

Couldn't figure out whether this is already fixed.

I try to insert a bunch of values that I query from my heatpump controller but for some of the values I get an A 400 Bad Request.
At first i thought that it is caused by the Umlauts but they work for some values, so I have no idea what causes this problem.
This is the object I send into the out node (copy&past from the debug panel, so not properly json formated ;-) ):

Vorlauf: 32.5
Rücklauf: 30.1
Rückl.-Soll: 15
Rückl.-Extern: 24.9
Heissgas: 28.8
Aussentemperatur: 34.3
Mitteltemperatur: 23.3
Warmwasser-Ist: 49
Warmwasser-Soll: 50
Wärmequelle-Ein: 24.2
Wärmequelle-Aus: 24.4
Mischkreis1-Vorlauf: 24.3
Mischkreis1-VL-Soll: 20
Solarkollektor: 5
Solarspeicher: 150
Externe-Energ.Quelle: 5
Vorlauf-max.: 64
Ansaug-VD: 29.5
Überhitzung: 5
Überhitzung-Soll: 6

And that's the error message:

Error: A 400 Bad Request error occurred: {"error":"unable to parse 'temperatures Ansaug-VD=29.5,Aussentemperatur=34.3,Externe-Energ.Quelle=5,Heissgas=28.8,Mischkreis1-VL-Soll=20,Mischkreis1-Vorlauf=24.3,Mitteltemperatur=23.3,Rückl.-Extern=24.9,Rückl.-Soll=15,Rücklauf=30.1,Solarkollektor=5,Solarspeicher=150,Vorlauf=32.5,Vorlauf-max.=64,Warmwasser-Ist=49,Warmwasser-Soll=50,Wärmequelle-Aus=24.4,Wärmequelle-Ein=24.2,Überhitzung=5,Überhitzung': invalid field format"}

If I reduce the object to the first 7 elements, it works, but when I include "Warmwasser-Ist" I get the error.
It doesnt's make sense to me at all.

as reported on node-influx #281

On the same vein as the error handing in #46 would it be ok to add result handling for successes of influx writes?

What I'm thinking is we add an output to influx batch with a .then on the writePoints which sends a message containing the 204 for the successful write.

Should I add this to the influx batch node or do you think it belongs in a separate node?

Hello,
is possible to write data to database with specific timestamp?

I'm handling with data from MQTT and sometimes data has a delay (due connection issue). They are written to database with same timestamp when they all come. My option is sending timestamp at MQTT message and use it for timestamp for Influx.

Thank you.

Something is wrong with the string I'm extracting from a Modbus buffer and transmitting to InfluxDB. The string looks right everywhere I debug it, but it's clear from InfluxDB queries that there is an issue with the string it records.
Here's my function as it sends to the db:

gwSerialString = "GTYA002018";
gwSerial = msg.responseBuffer.buffer.toString('utf8',0,16);
gwPower  = msg.responseBuffer.buffer.readInt32BE(28)/10;

msg.payload = [
    {
        measurement: "test",
        fields: {
            power: gwPower
        },
        tags:{
            serialString: gwSerialString,
            serial: gwSerial
        },
        timestamp: new Date()
    }
];
return msg;

Which appears in the debug window to return the same string as follows:

However, the InfluxDB does not see them the same way because trying to use the hard coded string works fine but using the one passed from the buffer.toString gives the following error when used in queries:
found �GTYA002018, expected identifier, string, number, bool
Which interestingly when pasted into this window and previewed I can now see an there's an extra character I haven't seen before before the expected string. This may be the clue as to where the issue is coming from.

I don't think anything is wrong with the batch uploader, but I cannot figure out where this string is getting messed up or how to "cleanse" it appropriately.

I am able to use the catch node to trap errors on a DB write, but how do I react to a successful operation? This node doesn't appear to set a status on success, and the write has no output. So for example, I have a flow that accepts an HTTP POST, formats the data, and writes it to InfluxDB. I can catch an error and send an appropriate HTTP response (4xx, 5xx, etc). But since I can't seem to trap a successful operation, I have no way to send a 200 OK response from my flow, other than to just trust that the operation succeeded. Maybe I'm missing something obvious?

When we added the new advanced query parameters, we didn't add nanosecond precision.

Aloha, thanks for your node contribution, I use it a lot! I am migrating some systems to Influx 2 and currently I am just using raw queries in Node-RED to interact with InfluxDB2 API. I was considering making a new set of contrib nodes specifically for InfluxDB 2, but wanted to get your take on it first. Do you think a new project is justified? Or perhaps you would like to roll support for V2 into this project? The official node client is out and works great.

Here are some references:

• INfluxDB 2.0 Overview
• InfluxDB v2.0 Documentation
• Influx API Service (full API Docs for v2)
• Official InfluxDB 2.0 JavaScript client

Dear Sir, may I know if there is a limitation to write data on influxDB?
I have this data that updates data every 100ms interval, what I want is to write data to influxDB using the influxdb out node at that speed interval (100ms), but I notice that on my database my data is in somehow 1-5s interval (randomly). I am just thinking that is there any limitations on the speed to write data in influxdb out?
Thank you.

Looking forward.

Currently we only support a single influxdb server configuration. Could support multiple hosts in a cluster configuration.

Mike Looking at influx library the IQueryOptions exposes the precision property making that available would make sense.

Also I could see some users wanting an epoch returned instead of default RFC3339 time stamp
However to get that you need to call epoch=[ns,u,µ,ms,s,m,h] in the HTTP/s api
https://docs.influxdata.com/influxdb/v1.2/tools/api/#query-string-parameters
But the influx library doesn't support this

https://groups.google.com/forum/#!searchin/node-red/influxdb%7Csort:relevance/node-red/BJp_MEhoZIE/QbfSZwbaCwAJ

This is supported in the API, we should add it to the batch output node similar to the query node.

Hello.
How Can I insert to InfluxDB from all topic mqtt?
I used Node -red Mqtt # -> influxDB out
What me need to insert in Measurement that would get the name of topics?

I have:

1. setup Docker on a Pi3 B+
2. successfully installed InfluxDB on Docker
3. successfully created a database and user
4. successfully inserted data via Raspbian command line
5. successfully installed Node-RED on Docker
6. successfully installed node-red-contrib-influxdb
7. successfully created a simple flow using Output Node example modified to talk to localhost:8086 and my DB name

executing the flow gives the standard message that it successfully injected timestamp, whereas looking at the debug messages, I get the ECONNREFUSED message

I suspect this connection issue is related to trying to connect from the Node-RED connection which is within its own containers to InfluxDB which is a different container.

Have you ever seen this?
Is this container to container trust?

I am trying to write an MQTT message to influxdb, and getting strange results. The message originates in OwnTracks, so it looks like so:

{
  "_type":"location",
  "acc":39,
  "alt":195,
  "batt":61,
  "conn":"w",
  "inregions":["Home"],
  "lat":55.0,
  "lon":37.0,
  "t":"p",
  "tid":"s5",
  "tst":1548442037,
  "vac":16,
  "vel":0
}

I am feeding it through the json parse node and straight into influxdb, and getting this error back:

Error: A 400 Bad Request error occurred: {"error":"unable to parse 'track _type=\"location\",acc=39,alt=195,batt=61,conn=\"w\",inregions=Home,lat=55.0,lon=37.0,t=\"u\",tid=\"s5\",tst=1548436524,vac=16,vel=0': invalid boolean"}

Other sources of data logged into influxdb, which are also blocks of json taken straight out of MQTT messages, do not have any problems, and I can't help but notice that something strange happened to inregions. Am I doing something wrong, or are arrays of strings not being handled correctly?

First, thank you for your influx node for nodered.

I have a question, is there a way to send multiple different measurements at the same time ? A few is ok but when I increase the number, the influxdb node will send me a timeout error.

Thank you for your help.

Benjamin

I find I am unable to push null values to the db. Sending a message {topic: some_topic, payload: null} results in the error
Error: {"error":"unable to parse 'tydwr value=null': invalid number"}
Here is a sample flow:
[
{
"id": "d1592df1.f9427",
"type": "inject",
"z": "e122f88c.3fc3c8",
"name": "",
"topic": "",
"payload": "",
"payloadType": "date",
"repeat": "",
"crontab": "",
"once": false,
"x": 189.5,
"y": 1141,
"wires": [
[
"8d921e96.1de868"
]
]
},
{
"id": "8d921e96.1de868",
"type": "function",
"z": "e122f88c.3fc3c8",
"name": "Null payload",
"func": "msg = {topic: "bedroom/radtemp", payload: null}\nreturn msg;",
"outputs": 1,
"noerr": 0,
"x": 377.5,
"y": 1140,
"wires": [
[
"e18ce4c0.21f96",
"eeb056ca.f8c358"
]
]
},
{
"id": "eeb056ca.f8c358",
"type": "debug",
"z": "e122f88c.3fc3c8",
"name": "",
"active": true,
"console": "false",
"complete": "true",
"x": 722.5,
"y": 957,
"wires": []
},
{
"id": "e18ce4c0.21f96",
"type": "influxdb out",
"z": "e122f88c.3fc3c8",
"influxdb": "55ac6d48.e3ffbc",
"name": "",
"measurement": "tydwr",
"x": 994,
"y": 469,
"wires": []
},
{
"id": "55ac6d48.e3ffbc",
"type": "influxdb",
"z": "",
"hostname": "127.0.0.1",
"port": "8086",
"protocol": "http",
"database": "tydwr",
"name": "localhost_tydwr"
}
]

There is no need to set this configuraiton in the backend since it is just a front end UI thing. Just show the advanced options if retentionPolicy and timeprecision are set to non-default values or not set.

When writing to influx using the batch node and providing a timestamp the readme says that it should be in milliseconds or should be a javascript Date object. It appears that in fact the time has to be provided in nanoseconds.

Hi,

Thanks for your node contribution, I use it a lot and really handy!

When using retention policies, the data does not end up in the measurements. If you check the log file you will see a 204 http return code that will say it is cached. But somehow it is not ending up into the DB.

Could you please check this?

See the following security advisory
https://www.npmjs.com/advisories/1065

Hey,

I am not sure if this is an issue with node-red-contrib-influxdb or node-red themselves, so I would ask you first.

The issue is; when you GET a flow which includes an influxdb node via node-red API GET localhost:1880/flow/{id}, the configuration of the influxdb node is not included in the response. So, if you POST the same flow in another node-red instance for example you get
[error] [influxdb out:2ebbdce2.4d8274] Configuration missing

However, the configuration can be found if you export the same flow via the node-red UI

An example of the missing config

{
   "id": "24b1fa97.98b266",
   "type": "influxdb",
   "z": "",
   "hostname": "127.0.0.1",
   "port": "8086",
   "protocol": "http",
   "database": "images",
   "name": ""
 } 

Thank you

The syntax for integer is ###i, but assigning such value in the payload would be treated as a string when passing it through the in node. Regular numeric values are treated as float in influxdb.

My JS Node (output gets fed into Influx node)

/* Example Payload

"{ "ServiceId": "urn:upnp-org:serviceId:TemperatureSensor1", "CurrentTemperature": 18.62, "DeviceId": 420, "DeviceName": "Utetemperatur", "Time": 1527632216, "RoomName": "Garage/Ute", "OldCurrentTemperature": 18.68, "Variable": "CurrentTemperature", "RoomId": 18, "DeviceType": "urn:schemas-micasaverde-com:device:TemperatureSensor:1" }"

*/

let pl = msg.payload;
if (pl === "testament_msg")
    return null;
pl = JSON.parse(pl);     

return {
    measurement: pl.DeviceName + " - " + pl.RoomName,
    payload: [{
        v: pl[pl.Variable],
    },{
        //serviceId: pl.ServiceId,
        deviceId: pl.DeviceId,
        deviceName: pl.DeviceName,
        roomId: pl.RoomId,
        roomName: pl.RoomName,
        //deviceType: pl.DeviceType            
    }]
};

Trying to write the following msg object:
{"measurement":"Kontakt Tvrum - TV-rum","payload":[{"v":1},{"deviceId":474,"deviceName":"Kontakt Tvrum","roomId":7,"roomName":"TV-rum"}],"_msgid":"1fbecdf8.afdac2"}

But getting
Error: A 400 Bad Request error occurred: {"error":"unable to parse 'Kontakt Tvrum - TV-rum,deviceId=474,deviceName=Kontakt\\ Tvrum,roomId=7,roomName=TV-rum v=1': invalid field format"}

When using a query node for multiple measurements the data is all concatenated together. So it isn't clear what segment of data belongs to which measurement.

Is there a way to make it return a dictionary for each measurement with the key being the measurement name?

Would be more useful if all the configuration entered in the UI could be altered via a new node.

This would mean that the flows could be scaled and the only difference between them would be the JSON passed to it at startup.

What do you think?

Cannot write NaN to influxdb.

Hello, I have a influxdb nonde inserting records into my InfluxDb Database, the problem is that sometimes, without apparent reason, I get:

Error: Internal Server Error
Error: No host available

What is the timeout? My hosts are always available!

Thank you

Mike, I'm running influxdb 1.1 on a RPI far a long while.
I've just added HTTPs to influx using a self singed cert as per
https://docs.influxdata.com/influxdb/v1.2/administration/https_setup/#setup-https-with-a-self-signed-certificate.

HTTPS is working, I can connect using Grafana and the old influx web ui.
I get the error
15 Jun 13:47:18 - [error] [influxdb out:6c73de65.8cb15] Error: self signed certificate

Feels like the issue is in node-influx?
Any thoughts.

All best
Lawrence

It is not possible to provide a retention policy in output node for insert data to influxDb or it may not be documented.

Hello!

Your contribution is really helping me! Thank you!
I'm already able to insert data into my Influx database, but there's something to be done so it would be perfet!

How would I add tags to the valeu?

The name of my measurement is "variaveis"
I want to add information on "value" --this is working
But also want to complete "desc" and "tag" values... and this is my problem. Would you help me?

Thanks in advance!

Hi Mike,

first of all thank you very much for your influxd node.
I would like to ask if it is supporting the influxdb line protocol? I have got everything to work but only when I format the data I am sending from my microcontroller as JSON. Then I have to put a JSON node between my receiving node and the influxdb node.
As I am a complete beginner it could be that I am doing something wrong.

Kind regards,
Tom

When using the influxdb out node to manually set a timestamp, the values are not written in the database (no error is returned or visible in the debug console).

Example code of the function before the node:

msg.payload = {
    time: new Date().getTime(),
    measurement1: 2,
    measurement1: 0
}

This is not written to the database. When leaving the time field out of the object, everything works as expected.

I am running the latest version of influxDB (1.2)

If i query the database, the timestamps are in GMT and do not consider the timezone of the client.
Also, i couldn't find a possibility to set the time zone manually.

Hello howto make a connection with influxdb with a self signed certificate in node-red? (-unsafeSsl)