telefonicaid / iotagent-json Goto Github PK

Create the scripts that generate the IOTA RPM as well as the linux services and configuration.

Split the code to move the one that uses a particular transport protocol (HTTP or MQTT) to a separate file (as it was already done for the Ultralight2.0 agent ).

With the MQTT IoT Agent this shouldn't be allowed, as two devices with the same ID and the same APIKey would use the same topics for its communications (and could not be distinguished by the IOTA).

The devices connecting with the IoTA using MQTT need to get configuration values from the IoTA.

The following mechanism is proposed:

• The device publish in a topic /{{apikey}}/{{deviceid}}/configuration/commands (where the IoTA is subscribed) a request for configuration.
• The IoTA publish in the topic /{{apikey}}/{{deviceid}}/configuration/values a JSON with the attributes of the CB requested by the device.

The current configuration retrieval format returns the attributes as an array of objects with the "name" and "value" attributes. This representation is two long, and it should be changed to a format similar to the one used to report measures, that is, a plain JSON object, with each attribute being the corresponding NGSI attribute.

The MQTT-based southbound API should use timestamps without separators in order to save bandwidth. This timestamps should be converted to regular timestamps before being sent to the Context Broker, and should be converted back again when requested with configuration commands.

The generic IOTA client expose their commands for its use in other IOTA Clients (commands related to device provisioning and context handling). They should be imported in this IOTA's command line client.

Is it done yet? If I register Led as passive attribute and I ask for it from Context Broker, IoTAMQTT shows this debug

time=2016-02-04T17:17:25.922Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Body:

{
    "entities": [
        {
            "type": "Device",
            "isPattern": "false",
            "id": "MQTT Device"
        }
    ],
    "attributes": [
        "Led"
    ]
}


time=2016-02-04T17:17:25.922Z | lvl=ERROR | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Tried to handle a query before the query handler was stablished.
time=2016-02-04T17:17:25.922Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [Error] handing request: [object Object]

After that, context broker replies with:

{
    "errorCode": {
        "code": "404",
        "reasonPhrase": "No context element found"
    }
}

Support the conversion of the P1 and B modules of Thinking Things when they come as attributes in the payload.

The subscriptions to MQTT topics are created when the devices are provisioned. They should also be recreated when the IoTA restarts.

Hi,
I am trying to provision device with:

(curl localhost:4041/iot/devices -s -S --header 'Content-Type: application/json' \
    --header 'Accept: application/json' --header 'Fiware-Service: mainflux' --header 'Fiware-ServicePath: /mqtt' -d @- | python -mjson.tool) <<EOF
{
    "devices": [ 
        { 
            "device_id": "DevID1", 
            "entity_name": "TheDevice1", 
            "entity_type": "DeviceType", 
            "attributes": [ 
                  { "id": "t", "name": "temperature", "type": "float" },
                  { "id": "h", "name": "humidity", "type": "float" }
            ],
            "lazy":[
                  { "id": "l", "name": "luminosity", "type": "percentage" }
            ],
            "commands": [
                  { "id": "t", "name": "turn", "type": "string" }
            ],
            "static_attributes": [
                  { "name": "serialID", "type": "02598347" }
            ]
        }
    ]
} 
EOF

And I am getting the error:

{
    "message": "Wrong syntax in request: Errors found validating request.",
    "name": "WRONG_SYNTAX"
}

Here is IOTA log:

drasko@Lenin:~/fiware/iotagent-mqtt$ node bin/iotagentMqtt.js 
time=2016-02-04T23:26:13.513Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.Global | msg=Activating IOT Agent NGSI Library.
time=2016-02-04T23:26:13.515Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.Global | msg=Falling back to Transient Memory registry for NGSI Library
time=2016-02-04T23:26:13.519Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Starting IoT Agent listening on port [4041]
time=2016-02-04T23:26:13.520Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Using config:

{
    "logLevel": "DEBUG",
    "contextBroker": {
        "host": "localhost",
        "port": "1026"
    },
    "server": {
        "port": 4041
    },
    "deviceRegistry": {
        "type": "memory"
    },
    "types": {},
    "service": "mainflux",
    "subservice": "/mqtt",
    "providerUrl": "http://localhost:4041",
    "deviceRegistrationDuration": "P1M",
    "defaultType": "Thing",
    "iotaVersion": "0.1.5"
}

time=2016-02-04T23:26:13.533Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Loading NGSI Contect server routes
time=2016-02-04T23:26:13.534Z | lvl=DEBUG | corr=n/a | trans=n/a | op=n/a | msg=Loading access validation Templates
time=2016-02-04T23:26:13.541Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentMQTT.Agent | msg=IoT Agent services activated
time=2016-02-04T23:26:13.556Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentMQTT.Agent | msg=MQTT Client connected
time=2016-02-04T23:26:13.556Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentMQTT.Agent | msg=Recreating subscriptions for all devices
time=2016-02-04T23:26:13.557Z | lvl=INFO | corr=n/a | trans=n/a | op=IOTAMQTT.Executable | msg=MQTT IoT Agent started
time=2016-02-04T23:26:41.729Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Request for path [/iot/devices] from [localhost:4041]
time=2016-02-04T23:26:41.730Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Body:

{
    "devices": [
        {
            "device_id": "DevID1",
            "entity_name": "TheDevice1",
            "entity_type": "DeviceType",
            "attributes": [
                {
                    "id": "t",
                    "name": "temperature",
                    "type": "float"
                },
                {
                    "id": "h",
                    "name": "humidity",
                    "type": "float"
                }
            ],
            "lazy": [
                {
                    "id": "l",
                    "name": "luminosity",
                    "type": "percentage"
                }
            ],
            "commands": [
                {
                    "id": "t",
                    "name": "turn",
                    "type": "string"
                }
            ],
            "static_attributes": [
                {
                    "name": "serialID",
                    "type": "02598347"
                }
            ]
        }
    ]
}


time=2016-02-04T23:26:41.733Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.RestUtils | msg=Errors found validating request: {"valid":false,"errors":[{"attribute":"additionalProperties","property":"devices.0.commands.0.id","actual":"t","message":"must not exist"},
"message":"must not exist"},{"attribute":"additionalProperties","property":"devices.0.attributes.1.id","actual":"h","message":"must not exist"},{"attribute":"additionalProperties","property":"devices.0.attributes.0.id","actual":"t","message":"must not exist"},{"attribute":"additionalProperties","property":"devices.0.lazy.0.id","actual":"l","message":"must not exist"}]}
time=2016-02-04T23:26:41.733Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [WRONG_SYNTAX] handing request: Wrong syntax in request: Errors found validating request.

Hello,
are there any examples how to send/recieve SSL/TLS encrypted messages?

Is this supported by IOTA-MQTT?

The current code makes a specific subscription for each topic of each device. This is not needed, as a wildcard could be used to group them all.

Hello,
what is the way to send commands or attribute updates to devices?

For example, user sets temp to 24 degrees via UI app, and this information must be propagated to the MQTT-connected actuator.

is there something like /1234/actuator01/commands or should we use SUB on /1234/actuator01/attributes?

If I understand well mosquitto_pub -t /1234/sensor01/configuration/commands -m '{ "type": "configuration", "fields": [ "sleepTime" ] }' is the way to send command IOTA -> Orion, but what is the correct way to send the command IOTA -> physical_device?

I have a device which subscribes to configuration value "luminosidad", but each time I deploy new software, I restart the IoTA MQTT (which is configured to memory instead of mongodb), I do the device provide again, and each time I do this I end up with more and more messages in Mosquitto topic /1234/MQTT/configuration/values:

{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}

This got triggered by a single change in Context Broker. It looks like each time the IoTA is restarted, it does subscribe to those values again, but it doesn't clear the subscriptions done before.

Is it intended? Is there a quick fix? I've not subscribed to Boton neither to prueba. Why do I get those?

It should be fixed to be in 24h format

If a malformed JSON is given to the agent, the parsing raises an exception and the agent crashes. This should be captured and a graceful error emitted.

Return the system time along with all the configuration requests, in an especial attribute named "DT".

The IOTA currently supports only direct provisioning of devices. Implement the support for devices that are part of a Configuration group.

The IoT Agent should support the existance of new attributes (not declared in the device provisioning) coming from the SouthBound.

P1 and C1 module translation was tested sending information through the multiattribute topic, but errors have been reported when the same values are sent using the single attribute one. Tests should be implemented to cover this functionality and, in case the error is reproducible, fix it.

the Iotagent-mqtt version request does not include the agent version:

 curl http://localhost:4052/iot/about
{"libVersion":"0.9.3","port":4052,"baseRoot":"/"}

There has been a change in how the B attribute must be interpreted. Here is an example of the old format: #0,B,0.0,0,0,0,0,0,BS,45.22,-1$ The new one should look like this: #0,B,3.98,1,0,1,1,0,-1$None

The new interpretation of values is:

Measure syntax:

#<id_i2c>,B,<voltaje>,<estado_activacion>,<hay_cargador>,<estado_carga>,<modo_carga>,<tiempo_desco_stack>,<sleep_time>$<sleep_condition>,

Answer syntax

#<id_i2c>,B,<estado_activacion_carga>,<modo_carga>,<tiempo_desconexion_stack>,<sleep_user>&<condition>,

Both formats must be maintained, differentiated using the number of fields.

When the Mosquitto shuts down or disconnects for whatever reason, once the mosquitto is up again, the subscriptions of the IOTA have been lost. It should, instead, try to recreate them when the connection is reacquired.

A sonar-project.properties file is needed in order to extract metrics for the repository. It should be filled following the example of other of our Node.js projects:

https://github.com/telefonicaid/fiware-pep-steelskin/blob/master/sonar-project.properties

Return the system time along with all the configuration requests, in an especial attribute named "CLK".

Create the command that starts the IoT Agent from the command line.

Currently, the IOTA is incapable of using any kind credentials to connect to MQTT. At least password credentials should be provided.

Add a new type of configuration command that subscribes to all the changes in a set of attributes of the correspondent entity.

Create a new transport binding for the JSON based protocol based on plain HTTP, along the lines of the one defined for Ultralight 2.0.

• Measure reporting.
• Commands.
• Configuration retrieval.

An alternative format as been proposed for the T1 module, called C1. In this alternative, the value of the C1 attribute will have the following format:

00D600070B000D24

Where the the string is divided in four groups of four hexadecimal characters, that represent, in this order:

• MMC: ('00D6' in the example), hexadecimal (to be transformed to decimal).
• MNC: (0007' in the example), hexadecimal (to be transformed to decimal).
• Lac: (0B00' in the example), hexadecimal.
• CellID: (0D24' in the example), hexadecimal.

Note that some of the values may need a hexadecimal to decimal conversion.

Hi,
I see that for provisioning in IoT Agents, they export and API endpoint which must be directly called from application.

From what I understand in this case, application directly calls IOTA API, and does not pass via Context Broker. Why? Isn't Context Broker a central component through all messages must flow?

I am asking this because I do not know if application can always be aware of IOTA location - when it is scaled and new Docker instances brought up they will have different IP address. How will then application discover the service (when we do not use PUB/SUB via Orion)?

Similar question goes to MQTT server - how will IOTA find horizontally scaled instances of MQTT server (hard-coded location is for one instance only, IMHO).

An extra comma in the tests of the B module made the module to require one parameter more than those needed. Both tests and code should be fixed.

Current documentation is very brief and some people is having problems understanding how the IoT Agent works. This points should be improved, at least:

• Explain briefly a typical IOTA workflow with a device, including provisioning and configuration provisioning.
• Explain the use of APIKeys and how to configure them.
• Explain the arcane ThinkingThings plugin (and add meanings to disable it using the configuration).
• Give some step-by-step guides to easily show how to start using the Agent.

A little back this year, the name of the attribute's attribute, 'id' was changed to 'object_id'. It seems part of the code remains as it was in the past, so the 'id' attribute is coming in the get provisioned device operation. It should be removed.

When the IoT Agent registers to the IoT Agent Manager, it declares a default resource, that will be used in every group provisioning to that agent. The IOTA should enforce that default resource in the group provisioning requests, instead of enforcing the empty string.

Under https://github.com/telefonicaid/iotagent-mqtt#-command-line-client please add that is needed a config file called "client-config.js" to use de CMD client with containing the basic config needed:

var config = {};

config.mqtt = {
    host: 'localhost',
    port: 1883,
    options: {
        keepalive: 0,
        connectTimeout: 60 * 60 * 1000
    }
};

config.device = {
    id: 'myDeviceId',
    apikey: '1234'
}

module.exports = config;

Configuration values are not retrieved correctly when asked via configuration topics. The problem is not detected in the tests, as a test validating the publishing format is also missing.

Create a new IOTA tester command interface, as the ones created for other IOTAs.

Can you please push the repo to the official Fiware DockerHub: https://hub.docker.com/u/fiware/.

While we can find LWM2M agent (https://hub.docker.com/r/fiware/lightweightm2m-iotagent/), MQTT agent is still missing.

Currently, the timestamp conversion mechanism do not work properly with aliased attributes. Tests should ensure that an attribute that is aliased to be a timestamp, will properly trigger timestamping mechanisms.

Change the name of the attribute generated in the device entity for a C1 device attribute to P1.

Hi,
I am looking at: https://github.com/telefonicaid/iotagent-mqtt#measure-reporting

I am wondering why is {{api-key}} part of URL and not part of the header or passed as a parameter?

What is this {{api-key}} and how do we obtain it, as it is not explained in the "Provisioning" section?

BR,
Drasko

This IoTA does not yet support lazy attributes but it support a very similar mechanism based on subscriptions and explicit requests from the device to retrieve information from the Context Broker. It should be easy to modify the configuration mechanism to allow lazy attributes (and consequently, commands), to be used with this IoTA.

The current behavior for RPMs is to replace the old config files with the newer ones in the code. This should be changed to avoid having to manually store de old configuration.

The resource value is not used in this IOTA, so a check should be performed to make sure it is allways empty (this is important, as the key to differentiate device configurations is by using the resource and apikey, so multiple different services could share the api key by generating random resources).

If a set of measures come to the IoTA with an attribute called TimeInstant, add that attribute as a metadata to all the other attributes.

The following message has appeared in the test executions, coming from an unknown dependency:

connections property is deprecated. Use getConnections() method

The dependency must be find and updated.

Current default config file has lots of possible attributes missing. To ease the use of the IOTA, a commented example config file should be provided.