Code Monkey home page Code Monkey logo

iotagent-json's Introduction

FIWARE IoT Agent for a JSON-based Protocol

FIWARE IoT Agents License: APGL Support badge
Quay badge Docker badge
Documentation badge CI Coverage Status Status CII Best Practices

An Internet of Things Agent for a JSON based protocol (with AMQP, HTTP and MQTT transports). This IoT Agent is designed to be a bridge between JSON and the NGSI interface of a context broker.

It is based on the IoT Agent Node.js Library. Further general information about the FIWARE IoT Agents framework, its architecture and the common interaction model can be found in the library's GitHub repository.

This project is part of FIWARE. For more information check the FIWARE Catalogue entry for the IoT Agents.

📚 Documentation 🎓 Academy quay.io 🐳 Docker Hub 🎯 Roadmap

Contents

Background

This IoT Agent is designed to be a bridge between an MQTT/HTTP+JSON based protocol and the FIWARE NGSI standard used in FIWARE. This project is based in the Node.js IoT Agent library. More information about the IoT Agents can be found within the library's GitHub repository.

A quick way to get started is to read the Step by step Manual.

As is the case in any IoT Agent, this one follows the interaction model defined in the Node.js IoT Agent Library, that is used for the implementation of the Northbound APIs. Information about the IoTAgent's architecture can be found on that global repository. This documentation will only address those features and characteristics that are particular to the JSON IoTAgent.

If you want to contribute to the project, check out the Development section and the Contribution guidelines.

Additional information about operating the component can be found in the Operations: logs and alarms document.

Install

Information about how to install the JSON IoTAgent can be found at the corresponding section of the Installation & Administration Guide.

A Dockerfile is also available for your use - further information can be found here

Usage

Information about how to use the IoT Agent can be found in the User & Programmers Manual.

The following features are listed as deprecated.

API

Apiary reference for the Configuration API can be found here More information about IoT Agents and their APIs can be found in the IoT Agent Library documentation.

The latest IoT Agent for JSON documentation is also available on ReadtheDocs

Contributing

If you'd like to contribute, start by searching through the issues and pull requests to see whether someone else has raised a similar idea or question.

Before contributing, please check out contribution guidelines

Testing

Mocha Test Runner + Should.js Assertion Library.

The test environment is preconfigured to run BDD testing style.

Module mocking during testing can be done with proxyquire

To run tests, type

npm test

Please have a look to extra information about functional tests in this specific document.

Requirements

All the tests are designed to test end-to-end scenarios, and there are some requirements for its current execution:

  • MQTT v5 broker (like mosquitto v1.6.7 server running)
  • MongoDB v4.x server running
  • RabbitMQ

To run requirements you can type:

   docker run -d -p 27017:27017 --hostname mongo --name mongo mongo:6.0.12
   docker run -d -p 1883:1883 --hostname mosquitto --name mosquitto eclipse-mosquitto:1.6.15
   docker run -d -p 5672:5672 --hostname rabbit --name rabbit rabbitmq:3.11.13

License

The IoT Agent for JSON is licensed under Affero General Public License (GPL) version 3.

© 2023 Telefonica Investigación y Desarrollo, S.A.U

Further information on the use of the AGPL open source license

Are there any legal issues with AGPL 3.0? Is it safe for me to use?

There is absolutely no problem in using a product licensed under AGPL 3.0. Issues with GPL (or AGPL) licenses are mostly related with the fact that different people assign different interpretations on the meaning of the term “derivate work” used in these licenses. Due to this, some people believe that there is a risk in just using software under GPL or AGPL licenses (even without modifying it).

For the avoidance of doubt, the owners of this software licensed under an AGPL-3.0 license wish to make a clarifying public statement as follows:

Please note that software derived as a result of modifying the source code of this software in order to fix a bug or incorporate enhancements is considered a derivative work of the product. Software that merely uses or aggregates (i.e. links to) an otherwise unmodified version of existing software is not considered a derivative work, and therefore it does not need to be released as under the same license, or even released as open source.

iotagent-json's People

Contributors

aarranz avatar alvarovega avatar anabelengp avatar cesarjorgemartinez avatar chandradeep11 avatar dcalvoalonso avatar deepshikhanec avatar dependabot[bot] avatar dmoranj avatar emmanuel-nau avatar fgalan avatar frbattid avatar gtorodelvalle avatar ivanhdzc avatar jagatjot avatar jason-fox avatar jmezzera avatar josueramosgomez avatar keshavsoni2511 avatar madhu-nec avatar manucarrace avatar mapedraza avatar mortonfox avatar mrutid avatar pawcik avatar rfgtid avatar rg2011 avatar taliaga avatar vgarciag avatar xavierval avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

iotagent-json's Issues

Improve documentation

Current documentation is very brief and some people is having problems understanding how the IoT Agent works. This points should be improved, at least:

  • Explain briefly a typical IOTA workflow with a device, including provisioning and configuration provisioning.
  • Explain the use of APIKeys and how to configure them.
  • Explain the arcane ThinkingThings plugin (and add meanings to disable it using the configuration).
  • Give some step-by-step guides to easily show how to start using the Agent.

Add complete commented config file

Current default config file has lots of possible attributes missing. To ease the use of the IOTA, a commented example config file should be provided.

Attribute alias don't work along with timestamp

Currently, the timestamp conversion mechanism do not work properly with aliased attributes. Tests should ensure that an attribute that is aliased to be a timestamp, will properly trigger timestamping mechanisms.

How to send commands to devices?

Hello,
what is the way to send commands or attribute updates to devices?

For example, user sets temp to 24 degrees via UI app, and this information must be propagated to the MQTT-connected actuator.

is there something like /1234/actuator01/commands or should we use SUB on /1234/actuator01/attributes?

If I understand well mosquitto_pub -t /1234/sensor01/configuration/commands -m '{ "type": "configuration", "fields": [ "sleepTime" ] }' is the way to send command IOTA -> Orion, but what is the correct way to send the command IOTA -> physical_device?

Add required files for integration with Sonar

A sonar-project.properties file is needed in order to extract metrics for the repository. It should be filled following the example of other of our Node.js projects:

https://github.com/telefonicaid/fiware-pep-steelskin/blob/master/sonar-project.properties

Passive reading of attributes.

Is it done yet? If I register Led as passive attribute and I ask for it from Context Broker, IoTAMQTT shows this debug

time=2016-02-04T17:17:25.922Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Body:

{
    "entities": [
        {
            "type": "Device",
            "isPattern": "false",
            "id": "MQTT Device"
        }
    ],
    "attributes": [
        "Led"
    ]
}


time=2016-02-04T17:17:25.922Z | lvl=ERROR | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Tried to handle a query before the query handler was stablished.
time=2016-02-04T17:17:25.922Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [Error] handing request: [object Object]

After that, context broker replies with:

{
    "errorCode": {
        "code": "404",
        "reasonPhrase": "No context element found"
    }
}

Support conversion of timestamp attributes from the South Bound

The MQTT-based southbound API should use timestamps without separators in order to save bandwidth. This timestamps should be converted to regular timestamps before being sent to the Context Broker, and should be converted back again when requested with configuration commands.

Create IOTA RPM

Create the scripts that generate the IOTA RPM as well as the linux services and configuration.

Create a new TT interpreted attribute to get codified Cell information

An alternative format as been proposed for the T1 module, called C1. In this alternative, the value of the C1 attribute will have the following format:

00D600070B000D24

Where the the string is divided in four groups of four hexadecimal characters, that represent, in this order:

  • MMC: ('00D6' in the example), hexadecimal (to be transformed to decimal).
  • MNC: (0007' in the example), hexadecimal (to be transformed to decimal).
  • Lac: (0B00' in the example), hexadecimal.
  • CellID: (0D24' in the example), hexadecimal.

Note that some of the values may need a hexadecimal to decimal conversion.

Change the configuration format

The current configuration retrieval format returns the attributes as an array of objects with the "name" and "value" attributes. This representation is two long, and it should be changed to a format similar to the one used to report measures, that is, a plain JSON object, with each attribute being the corresponding NGSI attribute.

Implement passive attributes (AKA lazy attributes)

This IoTA does not yet support lazy attributes but it support a very similar mechanism based on subscriptions and explicit requests from the device to retrieve information from the Context Broker. It should be easy to modify the configuration mechanism to allow lazy attributes (and consequently, commands), to be used with this IoTA.

Add support for configuration attributes

The devices connecting with the IoTA using MQTT need to get configuration values from the IoTA.

The following mechanism is proposed:

  • The device publish in a topic /{{apikey}}/{{deviceid}}/configuration/commands (where the IoTA is subscribed) a request for configuration.
  • The IoTA publish in the topic /{{apikey}}/{{deviceid}}/configuration/values a JSON with the attributes of the CB requested by the device.

Add generic IOTAgent functions to the client

The generic IOTA client expose their commands for its use in other IOTA Clients (commands related to device provisioning and context handling). They should be imported in this IOTA's command line client.

MQTT Agent should reject configurations that do specify the resource

The resource value is not used in this IOTA, so a check should be performed to make sure it is allways empty (this is important, as the key to differentiate device configurations is by using the resource and apikey, so multiple different services could share the api key by generating random resources).

Error on provisioning - "message":"must not exist"

Hi,
I am trying to provision device with:

(curl localhost:4041/iot/devices -s -S --header 'Content-Type: application/json' \
    --header 'Accept: application/json' --header 'Fiware-Service: mainflux' --header 'Fiware-ServicePath: /mqtt' -d @- | python -mjson.tool) <<EOF
{
    "devices": [ 
        { 
            "device_id": "DevID1", 
            "entity_name": "TheDevice1", 
            "entity_type": "DeviceType", 
            "attributes": [ 
                  { "id": "t", "name": "temperature", "type": "float" },
                  { "id": "h", "name": "humidity", "type": "float" }
            ],
            "lazy":[
                  { "id": "l", "name": "luminosity", "type": "percentage" }
            ],
            "commands": [
                  { "id": "t", "name": "turn", "type": "string" }
            ],
            "static_attributes": [
                  { "name": "serialID", "type": "02598347" }
            ]
        }
    ]
} 
EOF

And I am getting the error:

{
    "message": "Wrong syntax in request: Errors found validating request.",
    "name": "WRONG_SYNTAX"
}

Here is IOTA log:

drasko@Lenin:~/fiware/iotagent-mqtt$ node bin/iotagentMqtt.js 
time=2016-02-04T23:26:13.513Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.Global | msg=Activating IOT Agent NGSI Library.
time=2016-02-04T23:26:13.515Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.Global | msg=Falling back to Transient Memory registry for NGSI Library
time=2016-02-04T23:26:13.519Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Starting IoT Agent listening on port [4041]
time=2016-02-04T23:26:13.520Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Using config:

{
    "logLevel": "DEBUG",
    "contextBroker": {
        "host": "localhost",
        "port": "1026"
    },
    "server": {
        "port": 4041
    },
    "deviceRegistry": {
        "type": "memory"
    },
    "types": {},
    "service": "mainflux",
    "subservice": "/mqtt",
    "providerUrl": "http://localhost:4041",
    "deviceRegistrationDuration": "P1M",
    "defaultType": "Thing",
    "iotaVersion": "0.1.5"
}

time=2016-02-04T23:26:13.533Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentNGSI.ContextServer | msg=Loading NGSI Contect server routes
time=2016-02-04T23:26:13.534Z | lvl=DEBUG | corr=n/a | trans=n/a | op=n/a | msg=Loading access validation Templates
time=2016-02-04T23:26:13.541Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentMQTT.Agent | msg=IoT Agent services activated
time=2016-02-04T23:26:13.556Z | lvl=INFO | corr=n/a | trans=n/a | op=IoTAgentMQTT.Agent | msg=MQTT Client connected
time=2016-02-04T23:26:13.556Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentMQTT.Agent | msg=Recreating subscriptions for all devices
time=2016-02-04T23:26:13.557Z | lvl=INFO | corr=n/a | trans=n/a | op=IOTAMQTT.Executable | msg=MQTT IoT Agent started
time=2016-02-04T23:26:41.729Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Request for path [/iot/devices] from [localhost:4041]
time=2016-02-04T23:26:41.730Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Body:

{
    "devices": [
        {
            "device_id": "DevID1",
            "entity_name": "TheDevice1",
            "entity_type": "DeviceType",
            "attributes": [
                {
                    "id": "t",
                    "name": "temperature",
                    "type": "float"
                },
                {
                    "id": "h",
                    "name": "humidity",
                    "type": "float"
                }
            ],
            "lazy": [
                {
                    "id": "l",
                    "name": "luminosity",
                    "type": "percentage"
                }
            ],
            "commands": [
                {
                    "id": "t",
                    "name": "turn",
                    "type": "string"
                }
            ],
            "static_attributes": [
                {
                    "name": "serialID",
                    "type": "02598347"
                }
            ]
        }
    ]
}


time=2016-02-04T23:26:41.733Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.RestUtils | msg=Errors found validating request: {"valid":false,"errors":[{"attribute":"additionalProperties","property":"devices.0.commands.0.id","actual":"t","message":"must not exist"},
"message":"must not exist"},{"attribute":"additionalProperties","property":"devices.0.attributes.1.id","actual":"h","message":"must not exist"},{"attribute":"additionalProperties","property":"devices.0.attributes.0.id","actual":"t","message":"must not exist"},{"attribute":"additionalProperties","property":"devices.0.lazy.0.id","actual":"l","message":"must not exist"}]}
time=2016-02-04T23:26:41.733Z | lvl=DEBUG | corr=n/a | trans=n/a | op=IoTAgentNGSI.NorthboundServer | msg=Error [WRONG_SYNTAX] handing request: Wrong syntax in request: Errors found validating request.

Can provisioning be done from Orion?

Hi,
I see that for provisioning in IoT Agents, they export and API endpoint which must be directly called from application.

From what I understand in this case, application directly calls IOTA API, and does not pass via Context Broker. Why? Isn't Context Broker a central component through all messages must flow?

I am asking this because I do not know if application can always be aware of IOTA location - when it is scaled and new Docker instances brought up they will have different IP address. How will then application discover the service (when we do not use PUB/SUB via Orion)?

Similar question goes to MQTT server - how will IOTA find horizontally scaled instances of MQTT server (hard-coded location is for one instance only, IMHO).

Providing multiple times the same device ends with multiple events of subscription.

I have a device which subscribes to configuration value "luminosidad", but each time I deploy new software, I restart the IoTA MQTT (which is configured to memory instead of mongodb), I do the device provide again, and each time I do this I end up with more and more messages in Mosquitto topic /1234/MQTT/configuration/values:

{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120928Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}
{"Boton":" ","luminosidad":"40","prueba":"15","dt":"20160311T120929Z"}

This got triggered by a single change in Context Broker. It looks like each time the IoTA is restarted, it does subscribe to those values again, but it doesn't clear the subscriptions done before.

Is it intended? Is there a quick fix? I've not subscribed to Boton neither to prueba. Why do I get those?

Preserve the configuration files in updates

The current behavior for RPMs is to replace the old config files with the newer ones in the code. This should be changed to avoid having to manually store de old configuration.

Changes in the Thinking Things B attribute

There has been a change in how the B attribute must be interpreted. Here is an example of the old format: #0,B,0.0,0,0,0,0,0,BS,45.22,-1$ The new one should look like this: #0,B,3.98,1,0,1,1,0,-1$None

The new interpretation of values is:

Measure syntax:

#<id_i2c>,B,<voltaje>,<estado_activacion>,<hay_cargador>,<estado_carga>,<modo_carga>,<tiempo_desco_stack>,<sleep_time>$<sleep_condition>,

Answer syntax

#<id_i2c>,B,<estado_activacion_carga>,<modo_carga>,<tiempo_desconexion_stack>,<sleep_user>&<condition>,

Both formats must be maintained, differentiated using the number of fields.

P1 and C1 Thinking Thing modules do not work in single attribute mode

P1 and C1 module translation was tested sending information through the multiattribute topic, but errors have been reported when the same values are sent using the single attribute one. Tests should be implemented to cover this functionality and, in case the error is reproducible, fix it.

SSL/TLS Encryption

Hello,
are there any examples how to send/recieve SSL/TLS encrypted messages?

Is this supported by IOTA-MQTT?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.