• Overview
• Build & Install
• API Overview
  • About API
  • Device Provisioning API
  • Configuration API
• Advanced Topics
  • Secured access to the Context Broker
  • Data mapping plugins
  • Old IoTAgent data migration
• Testing
• Development Documentation

This project aims to provide a Node.js module to enable IoT Agent developers to build custom agents for their devices that can easily connect to NGSI Context Brokers (such as Orion ).

An IoT Agent is a component that lets groups of devices send their data to and be managed from a FIWARE NGSI Context Broker using their own native protocols. IoT Agents should also be able to deal with security aspects of the Fiware platform (authentication and authorization of the channel) and provide other common services to the device programmer.

There is more information about specific topics in the following documents:

• User manual: library reference for IoTA implementation. Contains the complete list of library functions and implementation topics.
• Installation and configuration guide: information for the configuration of the library.
• Operations manual: guide of logs and alarms raised by the library.
• IoTA Design HowTo: step-by-step howto about how to implement an IoT Agent.

This project is part of FIWARE. Check also the FIWARE Catalogue entry for the IoTAgents

Each Device will be mapped as an Entity associated to a Context Provider: the Device Id will be mapped by default to the entity ID and the type of the entity will be selected by the IoT Agent in a protocol-dependent way (e.g: with different URLs for different types). Both the name and type will be configurable by the user, either by type configuration or with the device preprovisioning.

Each of the measures obtained from the device should be mapped to a different attribute. The name and type of the attribute will be configured by the user (globally for all the types in the IoT Agent configuration or in a per device basis preprovisioning the devices). Device measures can have three different behaviors:

• Active attributes: are measures that are pushed from the device to the IoT agent. This measure changes will be sent to the Context Broker as updateContext requests over the device entity. NGSI queries to the context broker will be resolved in the Broker database.

• Lazy attributes: some sensors will be passive, and will wait for the IoT Agent to request for data. For those measures, the IoT Agent will register itself in the Context Broker as a Context Provider (for all the lazy measures of that device), so if any component asks the Context Broker for the value of that sensor, its request will be redirected to the IoT Agent (that behaves as a NGSI10 Context Provider). This operation will be synchronous from the customer perspective: the Context Broker won't return a response until de device has returned its response to the IoT Agent.

• Commands: in this case, the interaction will begin by setting an attribute in the device's entity, for which the IoT Agent will be regitered as CP. The IoT Agent will return an immediate response to the Context Broker, and will be held responsible of contacting the device to perform the command itself, updating special status and info attributes in the entity as soon as it has any information of the command progress.

The following sequence diagram shows the different NGSI interactions an IoT Agent makes with the Context Broker, explained in the following subsections (using the example of a OMA Lightweight M2M device).

Be aware that the IoT Agents are only required to support NGSI10 operations updateContext and queryContext in their standard formats (currently in JSON format; XML deprecated) but will not answer to NGSI9 operations (or NGSI convenience operations of any kind).

In order for a device to connect to the IoTA, the device should be provisioned (although there may be occasions where this registration is not needed). The provision process is meant to provide the IoTA with the following information:

• Entity translation information: information about how to convert the data coming from the South Bound into NGSI information. This includes things as the entity name and type and the name and type of all the attributes that will be created in the entity. This includes the service and subservice the entity belongs to.

• Southbound protocol identification: attributes that will identify a particular device when a new measure comes to the Southbound (typically the Device ID and API Key).

• Security information: trust token for the devices to be inserted in a PEP Protected Context Broker.

• Other information: as timezone or alternative Context Brokers.

In order to provide this information, the IoTAgent Northbound API provides two resources: Device and Configuration provisioning.

Configurations may be used when a set of similar devices will be connected to the IoTA, to avoid provisioning the same set of information for every device. Custom APIKeys can be only provided with the use of Configurations for device groups. When a device is provisioned, it is assigned to a configuration if there is one that matches its type, its service and its subservice. In that case, all the default information in the Configuration is merged with the device information to create the definitive Device object that will be stored in the system.

Particular IoT Agents may support autoregistration of devices into configurations, if enough information is given from the Southbound.

Configurations are meant to be a mean of simplifying the device provisioning for groups of very similar devices. Considering that different groups of devices may be created in the same subservice that may require different configurations, multiple configurations are allowed for each subservice. Considering the key in the association between Device and Configuration was the triplet (service, subservice, type), all of these elements are considered mandatory.

This statement doesn't hold true for older IoT Agents, though. In older versions of the IoT Agents, each device configuration was assigned to a particular subservice and just one configuration was allowed per subservice, so the relation between a Device and a Configuration didn't need the type to discriminate between Configurations. That's why for those agents, type was not a mandatory parameter.

In order to allow backward-compatibility with those agents, the IoT Agent Library now implement a compatibility mode: the Single Configuration Mode, that makes the agent behave like the old agents. In this mode:

• Each Subservice can contain just one Configuration. If a second Configuration is created for a Subservice, an error is raised.

• Each Device provisioned for a Subservice is automatically assigned to the Subservice one Configuration if there is any.

This compatibility has to be set for the whole IoT Agent, and there is no option of having both modes simultaneously running. Transitions from one mode to the other should be made with care, and may involve data migration.

Whenever a device is registered, the IoT Agent reads the device's entity information from the request or, if that information is not in the request, from the default values for that type of device. Among this information, there should be the list of device attributes that will be considered lazy (or passive). With this information, the IoT Agent sends a new registerContext request to the Context Broker, registering itself as ContextProvider of all the lazy attributes for the device's entity. The registrationId is then stored along the other device information inside the IoT Agent device registry.

As NGSI9 does not allow the context registrations to be removed, when the device is removed from the IoT Agent, the registration is updated to an expiration date of 1s, so it is effectively disabled. Once it has been disabled, the device is removed from the IoT Agent's internal registry.

When a request for data from a lazy attribute arrives to the Context Broker, it forwards the request to the Context Provider of that entity, in this case the IoT Agent. The IoT Agent will in turn ask the device for the information needed, transform that information to a NSGI format and return it to the Context Broker. The latter will the forward the response to the caller, transparently.

IMPORTANT NOTE: at the present moment, commands (both push and poll) are supported only in the case of explictely provisioned agents. For autoprovisioned agents commands are not currently supported, although an issue has been created about this functionality.

Commands are modelled as updates over a lazy attribute. As in the case of the lazy attributes, updates over a command will be forwarded by the Context Broker to the IoT Agent, that will in turn interact with the device to perform the requested action. Parameters for the command will be passed inside the command value.

There are two differences with the lazy attributes:

• First of all, for every command defined in a device, two new attributes are created in the entity with the same name as the command but with a prefix:

  • '_info': this attribute reflect the current execution status of the command. When a command request is issued by the Context Broker, the IoT Agent library generates this attribute with 'PENDING' value. The value of this attribute will be changed each time a command error or result is issued to the IoT Agent.
  • '_result': this attribute reflect the result of the execution of the defined command.

• Commands can also be updated when new information about its execution arrives to the agent. This information will be mapped to the command's utility attributes _info and _result leaving alone the command attribute itself. The values for this attributes are stored locally in the Context Broker (instead of being redirected with the Context Provider operations).

There are two types of commands:

• Push commands: when a command of this type arrives to the IoTAgent, the IoTAgent will immediately forward the command request to the device, translating the request to the proper protocol (that will depend on the type of IoTAgent). The library implement this kind of commands by offering a set functions that can be used to set an IoTAgent-specific handler for incoming commands. In order for this type of commands to work properly, the devices must be preprovisioned with an endpoint of the proper protocol, where it can be accessed by the IoTAgent who pushes de commits.

• Poll commands: polling commands are meant to be used on those cases where the device can't be online the whole time waiting for commands. In this case, the IoTAgents must store the received commands, offering a way for the device to retrieve the pending commands upon connection. To enable this feature, the Library offers a set of functions to manage command storage, and a mechanism to automatically store incoming commands for those devices marked as 'polling devices'.

The distinction between push and poll commands will be made based on the presence of a polling flag in the device provisioning data. The details on how this flag is derived for provisioning data would depend on the particular IOT Agent implementation using this libray (in other words, there isn't any standard way of doing so). The default option (with the flag with value false or not present) is to use push commands (as they were the only ones available until the latest versions).

Polling commands could be subjected to expiration: two configuration properties pollingExpirationandpollingDaemonFrequency` can be set to start a daemon that will remove expired commands from the DB if the device is taking too much to pick them up. See the configuration section for details.

The library does not deal with protocol transformation or South Bound communications for neither of the command types (that's the task for those specific IoTAgents using the library).

Whenever a device proactively sends a message to the IoT Agent, it should tranform its data to the appropriate NGSI format, and send it to the Context Broker as an updateContext request.

These are the features an IoT Agent is supposed to expose (those not supported yet by this library are marked as PENDING):

• Device registration: multiple devices will be connected to each IoT Agent, each one of those mapped to a CB entity. The IoT Agent will register itself as a Context Provider for each device, answering to requests and updates on any lazy attribute of the device.

• Device information update: whenever a device haves new measures to publish, it should send the information to the IoT Agent in its own native language. This message should , in turn, should be sent as an updateContext request to the Context Broker, were the measures will be updated in the device entity.

• Device command execution and value updates: as a Context Provider, the IoT Agent should receive update operations from the Context Broker subscriptions, and relay them to the corresponding device (decoding it using its ID and Type, and other possible metadata). This commands will arrive as updateContext operations redirected from the Context Broker to the IoT Agent (Command execution PENDING; value updates available).

• Device management: the IoT Agent should offer a device repository where the devices can be registered, holding data needed for the connection to the Context Broker as the following: service and subservice for the device, API Key the device will be using to connect to the IoT Agent, Trust token the device will be using to retrieve the Keystone token to connect to the Context Broker.

• Device provisioning: the IoT Agent should offer an external API to make a preprovision of any devices. This preprovision should enable the user to customize the device`s entity name and type as well as their service information.

• Type configuration: if a device is registered without a preregistration, only its id and type attributes are mandatory. The IoT Agent should provide a mechanism to provide default values to the device attributes based on its type.

Almost all of these features are common for every agent, so they can be abstracted into a library or external module. The objective of this project is to provide that abstraction. As all this common tasks are abstracted, the main task of the concrete IoT Agent implementations will be to map between the native device protocol and the library API.

The following figure offers a graphical example of how a COAP IoT Agent work, ordered from the registration of the device to a command update to the device.

As part of the device to entity mapping process the IoT Agent creates and updates automatically a special timestamp. This timestamp is represented as two different properties of the mapped entity::

• An attribute metadata named TimeInstant per dynamic attribute mapped, which captures as an ISO8601 timestamp when the associated measurement (represented as attribute value) was observed.

• An entity attribute named TimeInstant which captures as an ISO8601 timestamp when the last measurement received from the device was observed.

If no information about the measurement timestamp is received by the IoTAgent, the arrival time of the measurement will be used to generate a TimeInstant for both the entity and the attribute's metadata.

Take into account that:

• the timestamp of different attributes belonging to the same measurement record may not be equal.
• the arrival time and the measurement timestamp will not be the same in the general case.

E.g.: in the case of a device that can take measurements every hour of both temperature and humidity and sends the data once every day, at midnight, the TimeInstant reported for each measurement will be the hour when that measurement was observed (e.g. 4:00 PM), while all the measurements will have an arrival time around midnight. If no timestamps were reported with such measurements, the TimeInstant attribute would take those values around midnight.

This functionality can be turned on and off through the use of the timestamp configuration flag (described in the configuration).

Given the aforementioned requirements, there are some aspects of the implementation that were chosen, and are particularly under consideration:

• Aside from its configuration, the IoT Agent Lib is considered to be stateless. To be precise, the library mantains a state (the list of entities/devices whose information the agent can provide) but that state is considered to be transient. It's up to the particular implementation of the agent to consider whether it should have a persistent storage to hold the device information (so the internal list of devices is read from a DB) or to register the devices each time a device sends a measure. To this extent, two flavours of the Device Registry has been provided: a transient one (In-memory Registry) and a persistent one (based in MongoDB).
• The IoT Agent does not care about the origin of the data, its type or structure. The mapping from raw data to the entity model, if there is any, is a responsability of the particular IoT Agent implementation, or of another third party library.

Information about how to configure the Library can be found at the corresponding section of the Installation & Administration Guide.

This library has no packaging or build processes. Usage of the library is explained in the User & Programmers Manual.

The library provides a simple operation to retrieve information about the library and the IoTA using it. A GET request to the /iot/about path, will show a payload like the following:

{
“version”:”0.5.2”,
“libVersion”:”0.8.4”,
"port":4041,
"baseRoot":"/"
}

the version field will be read from the iotaVersion field of the config, if it exists.

The IoT Agents offer a provisioning API where devices can be preregistered, so all the information about service and subservice mapping, security information and attribute configuration can be specified in a per device way instead of relaying on the type configuration. The following section specifies the format of the device payload; this will be the payload accepted by all the write operations and that will be returned by all the read operations. Take care of the exception of the POST operation: in this case, the device objects must be specified as an array, as multiple devices can be provided simultaneusly for the same service.

Two parameters in this payload are given a special treatment: service and subservice. This two parameters are needed to fill the fiware-service and fiware-servicepath mandatory headers that will be used in the interactions with the Context Broker. This parameters should not be passed along with the rest of the body, but they will be taken from the same headers, as received by the Device Provisioning API (this two headers are, thus, mandatory both for incoming and outgoing requests).

Note that there is a 1:1 correspondence between payload fields and DB fields (but using a different capitalization, e.g. service_path vs. servicePath).

The next table shows the information held in the Device resource. The table also contains the correspondence between the API resource fields and the same fields in the database model.

In the device model there are three list of attributes that can be declared: attributes, lazy and commands. All of them have the same syntax, an object containing the following attributes:

• object_id (optional): name of the attribute as coming from the device.
• name (mandatory): id of the attribute in the target entity in the Context Broker.
• type (mandatory): name of the type of the attribute in the target entity.

Some transformation plugins also allow the use of the following optional attributes:

• expression: indicates that the value of the target attribute will not be the plain value or the measurement, but an expression based on a combination of the reported values. See the Expression Language definition for details
• entity_name: the presence of this attribute indicates that the value will not be stored in the original device entity but in a new entity with an ID given by this attribute. The type of this additional entity can be configured with the entity_type attribute. If no type is configured, the device entity type is used instead. Entity names can be defined as expressions, using the Expression Language definition.
• entity_type: configures the type of an alternative entity.
• reverse: add bidirectionality expressions to the attribute. See the bidirectionality transformation plugin in the Data Mapping Plugins section for details.

See the transformation plugins Section for more details.

Provision a new device in the IoT Agent's device registry. Takes a Device in JSON format as the payload.

Returns:

• 200 OK if successful.
• 500 SERVER ERROR if there was any error not contemplated above.

Payload example:

{
    "devices": [ 
        { 
            "device_id": "DevID1", 
            "entity_name": "TheDevice1", 
            "entity_type": "DeviceType", 
            "attributes": [ 
                  { "object_id": "t", "name": "temperature", "type": "float" },
                  { "object_id": "h", "name": "humidity", "type": "float" }
            ],
            "lazy":[
                  { "object_id": "l", "name": "luminosity", "type": "percentage" }
            ],
            "commands": [
                  { "object_id": "t", "name": "turn", "type": "string" }
            ],
            "static_attributes": [
                  { "name": "serialID", "type": "02598347" }
            ]
        }
    ]
}

Returns a list of all the devices in the device registry with all its data.

Query parameters:

• limit: if present, limits the number of devices returned in the list.
• offset: if present, skip that number of devices from the original query.

Returns:

• 200 OK if successful, and the selected Device payload in JSON format.
• 404 NOT FOUND if the device was not found in the database.
• 500 SERVER ERROR if there was any error not contemplated above.

Example of return payload:

{
  "count": 2,
  "devices": [
    {
      "device_id": "DevID0",
      "service": "ServiceTest",
      "service_path": "/testSubservice",
      "entity_name": "TheDevice0",
      "entity_type": "DeviceType",
      "attributes": [
        {
          "type": "float",
          "name": "temperature",
          "object_id": "t"
        },
        {
          "type": "float",
          "name": "humidity",
          "object_id": "h"
        }
      ],
      "lazy": [],
      "static_attributes": [],
      "internal_attributes": []
    },
    {
      "device_id": "DevID1",
      "service": "ServiceTest",
      "service_path": "/testSubservice",
      "entity_name": "TheDevice1",
      "entity_type": "DeviceType",
      "attributes": [
        {
          "type": "float",
          "name": "temperature",
          "object_id": "t"
        },
        {
          "type": "float",
          "name": "humidity",
          "object_id": "h"
        }
      ],
      "lazy": [
        {
          "type": "percentage",
          "name": "luminosity",
          "object_id": "l"
        }
      ],
      "static_attributes": [
        {
          "type": "02598347",
          "name": "serialID"
        }
      ],
      "internal_attributes": []
    }

Returns all the information about a particular device.

Returns:

• 200 OK if successful, and the selected Device payload in JSON format.
• 404 NOT FOUND if the device was not found in the database.
• 500 SERVER ERROR if there was any error not contemplated above.

Example of return payload:

{
  "device_id": "DevID1",
  "service": "ServiceTest",
  "service_path": "/testSubservice",
  "entity_name": "TheDevice1",
  "entity_type": "DeviceType",
  "attributes": [
    {
      "object_id": "t",
      "name": "temperature",
      "type": "float"
    },
    {
      "object_id": "h",
      "name": "humidity",
      "type": "float"
    }
  ],
  "lazy": [
    {
      "object_id": "l",
      "name": "luminosity",
      "type": "percentage"
    }
  ],
  "static_attributes": [
    {
      "name": "serialID",
      "type": "02598347"
    }
  ],
  "internal_attributes": []
}

Remove a device from the device registry. No payload is required or received.

Returns:

• 200 OK if successful, with no payload.
• 404 NOT FOUND if the device was not found in the database.
• 500 SERVER ERROR if there was any error not contemplated above.

Changes the stored values for the device with the provided Device payload. Neither the name, the type nor the ID of the device can be changed using this method (as they are used to link the already created entities in the CB to the information in the device). Service and servicepath, being taken from the headers, can't be changed also.

Returns:

• 200 OK if successful, with no payload.
• 404 NOT FOUND if the device was not found in the database.
• 500 SERVER ERROR if there was any error not contemplated above.

Payload example:

{ 
    "attributes": [ 
          { "object_id": "t", "name": "temperature", "type": "float" },
          { "object_id": "h", "name": "humidity", "type": "float" },
          { "object_id": "p", "name": "pressure", "type": "float" }
    ],
    "lazy":[
          { "object_id": "l", "name": "luminosity", "type": "percentage" }
    ],
    "commands": [
          { "object_id": "t", "name": "turn", "type": "string" }
    ],
    "static_attributes": [
          { "name": "serialID", "type": "02598347" }
    ]
}

For some services, there will be no need to provision individual devices, but it will make more sense to provision different device groups, each of one mapped to a different type of entity in the context broker. How the type of entity is assigned to a device will depend on the Southbound technology (e.g.: path, port, APIKey...). Once the device has an assigned type, its configuration values can be extracted from those of the type.

The IoT Agents provide two means to define those device groups:

• Static Type Configuration: configuring the ngsi.types property in the config.js file.
• Dynamic Configuration API: making use of the API URLS in the configuration URI, /iot/services. Please, note that the configuration API manage servers under an URL that requires the server.name parameter to be set (the name of the IoT Agent we are using). If no name is configured default is taken as the default one.

Both approaches provide the same configuration information for the types (and they, in fact, end up in the same configuration collection).

Both approaches are better described in the sections bellow.

The following sections show the available operations for the Configuration API. Every operation in the API require the fiware-service and fiware-servicepath to be defined; the operations are performed in the scope of those headers. For the list case, the special wildcard servicepath can be specified, '/*'. In this case, the operation applies to all the subservices of the service given by the fiware-service header.

For every Device Group, the pair (resource, apikey) must be unique (as it is used to identify which group to assign to which device). Those operations of the API targeting specific resources will need the use of the resource and apikey parameters to select the apropriate instance.

Note that there is a 1:1 correspondence between payload fields and DB fields (but with some changes in the attribute naming; e.g.: subservice -> service_path).

The next table shows the information held in the Device Group resource. The table also contains the correspondence between the API resource fields and the same fields in the database model.

Creates a set of device groups for the given service and service path. The service and subservice information will taken from the headers, overwritting any preexisting values.

Body params:

• services: list of device groups to create. Each one adheres to the Device Group Model.

E.g.:

{
	"services": [
	{
	    "resource": "/deviceTest",
	    "apikey": "801230BJKL23Y9090DSFL123HJK09H324HV8732",
	    "type": "Light",
	    "trust": "8970A9078A803H3BL98PINEQRW8342HBAMS",
	    "cbHost": "http://unexistentHost:1026",
	    "commands": [
	        {
	            "name": "wheel1",
	            "type": "Wheel"
	        }
	    ],
	    "lazy": [
	        {
	            "name": "luminescence",
	            "type": "Lumens"
	        }
	    ],
	    "active": [
	        {
	            "name": "status",
	            "type": "Boolean"
	        }
	    ]
	}
	]
}

Returns:

• 200 OK if successful, with no payload.
• 400 MISSING_HEADERS if any of the mandatory headers is not present.
• 400 WRONG_SYNTAX if the body doesn't comply with the schema.
• 500 SERVER ERROR if there was any error not contemplated above.

Retrieves device groups from the database. If the servicepath header has de wildcard expression, '/*', all the subservices for the service are returned. The specific subservice parameters are returned in any other case.

Returns:

• 200 OK if successful, returning a device group body.
• 400 MISSING_HEADERS if any of the mandatory headers is not present.
• 500 SERVER ERROR if there was any error not contemplated above.

Modifies the information for a device group configuration, identified by the resource and apikey query parameters. Takes a device group body as the payload. The body does not have to be complete: for incomplete bodies, just the existing attributes will be updated

E.g.:

{
    "trust": "8970A9078A803H3BL98PINEQRW8342HBAMS",
    "cbHost": "http://anotherUnexistentHost:1026"
}

Returns:

• 200 OK if successful, returning the updated body.
• 400 MISSING_HEADERS if any of the mandatory headers is not present.
• 500 SERVER ERROR if there was any error not contemplated above.

Removes a device group configuration from the DB, specified by the resource and apikey query parameters.

Returns:

• 200 OK if successful.
• 400 MISSING_HEADERS if any of the mandatory headers is not present.
• 500 SERVER ERROR if there was any error not contemplated above.

The IoT Agent can be configured to expect certain kinds of devices, with preconfigured sets of attributes, service information, security information and other attributes. The types attribute of the configuration is a map, where the key is the type name and the value is an object containing all the type information. Each type can has the following information configured:

• service: service of the devices of this type.
• subservice: subservice of the devices of this type.
• active: list of active attributes of the device. For each attribute, its name and type must be provided.
• lazy: list of lazy attributes of the device. For each attribute, its name and type must be provided.
• commands: list of commands attributes of the device. For each attribute, its name and type must be provided.
• internalAttributes: optional section with free format, to allow specific IoT Agents to store information along with the devices in the Device Registry.
• staticAttributes: this array of attributes will be added to every entity of this type 'as is'.
• trust: trust token to use for secured access to the Context Broker for this type of devices (optional; only needed for secured scenarios).
• cbHost: Context Broker host url. This option can be used to override the global CB configuration for specific types of devices.

For access to instances of the Context Broker secured with a PEP Proxy, an authentication mechanism based in Keystone Trust tokens is provided. A Trust token is a long-term token that can be issued by any user to give another user permissions to impersonate him with a given role in a given project.

For the authentication mechanisms to work, the authentication attribute in the configuration has to be fully configured, and the authentication.enabled subattribute should have the value true.

When the administrator of a service is configuring a set of devices or device types in the IoT Agent to use a secured Context Broker, he should follow this steps:

• First, a Trust token should be requested to Keystone, using the service administrator credentials, the role ID and the IOT Agent User ID. The Trust token can be retrieved using the following request (shown as a curl command):

curl http://${KEYSTONE_HOST}/v3/OS-TRUST/trusts \
    -s \
    -H "X-Auth-Token: $ADMIN_TOKEN" \
    -H "Content-Type: application/json" \
    -d '
{
    "trust": {
        "impersonation": false,
        "project_id": "'$SUBSERVICE_ID'",
        "roles": [
            {
                "id": "'$ID_ROLE'"
            }
        ],
        "trustee_user_id": "'$ID_IOTAGENT_USER'",
        "trustor_user_id": "'$ID_ADM1'"
    }
}'

• Every device or type of devices configured to use a secured Context Broker must be provided with a Trust Token in its configuration.
• Before any request is sent to a secured Context Broker, the IoT Agent uses the Trust token to generate a temporary access token, that is attached to the request (in the X-Auth-token header).

Apart from the generation of the trust, the use of secured Context Brokers should be transparent to the user of the IoT Agent.

The IoT Agent Library provides a plugin mechanism in order to facilitate reusing code that makes small transformations on incoming data (both from the device and from the context consumers). This mechanism is based in the use of middlewares, i.e.: small pieces of code that receive and return an entity, making as many changes as they need, but taking care of returning a valid entity, that can be used as the input for other middlewares; this way, all those pieces of code can be chained together in order to make all the needed transformations in the target entity.

There are two kinds of middlewares: updateContext middlewares and queryContext middlewares. The updateContext middlewares are applied before the information is sent to the Context Broker, modifiying the entity before it is sent to Orion. The queryContext middlewares are applied on the received data, whenever the IoT Agent queries the Context Broker for information. I.e.: both middlewares will be automatically applied whenever the update() or query() functions are called in the library.

All the middlewares have the opportunity to break the chain of middleware applications by calling the callback() with an error object (the usual convention). If any of the updateContext middlewares raise an error, no request will be sent to the Context Broker. On the other hand, the queryContext request is always performed, but the call to the query() function will end up in an error if any of the queryContext middlewares report an error.

All the middlewares have the same signature:

function middlewareName(entity, typeInformation, callback) {}

The arguments for any middleware are the NGSI data over which it can operate:

• An updateContext payload in the case of an updateContext middleware and a queryContext payload otherwise;
• a typeInformation object containing all the information about the device stored during registration.
• and the customary callback parameter, with the usual meaning. It's really important for the library user to call this callback, as failing to do so may hang the IoT Agent completely. The callback must be called with the an optional error in the first argument and the same arguments recieved (potentially modified) as the following.

In order to manage the middlewares to the system, the following functions can be used:

• addUpdateMiddleware: adds an updateContext middleware to the stack of middlewares. All the middlewares will be applied to every call to the update() function. The final payload of the updateContext request will be the result of applying all this middlewares in the order they have been defined.

• addQueryMiddleware: adds a queryContext middleware to the stack of middlewares. All the middlewares will be applied to every call to the query() function.

• resetMiddlewares: remove all the middlewares from the system.

Usually, the full list of middlewares an IoT Agent will use would be added in the IoTAgent start sequence, so they should not change a lot during the IoT lifetime.

The library provides some plugins out of the box, in the dataPlugins collection. In order to load any of them, just use the addQueryMiddleware and addUpdateMiddleware functions with the selected plugin, as in the example:

var iotaLib = require('iotagent-node-lib');

iotaLib.addUpdateMiddleware(iotaLib.dataPlugins.compressTimestamp.update);
iotaLib.addQueryMiddleware(iotaLib.dataPlugins.compressTimestamp.query);

This plugins change all the timestamp attributes found in the entity, and all the timestamp metadata found in any attribute, from the basic complete calendar timestamp of the ISO8601 (e.g.: 20071103T131805) to the extended complete calendar timestamp (e.g.: +002007-11-03T13:18). The middleware expects to receive the basic format in updates and return it in queries (and viceversa, receive the extended one in queries and return it in updates).

In the Device provision, an id can be specified for each attribute, along with its name. The Id can be used then as the left part of a mapping from attribute names in the south bound to attribute names in the North Bound. If the id and name attributes are used in this way, this plugin makes the translation from one to the other automatically.

This plugin allows for the creation of Event attributes, i.e.: attributes whose value will be the timestamp of its inclusion in the system, regardless of the value they carried in the Southbound API. If this plugin is active, all the events in the IoT Agent with the configured type name will be marked as events. The event name can be configured in the config.eventType attribute.

This plugin processes the entity attributes looking for a TimeInstant attribute. If one is found, the plugin add a TimeInstant attribute as metadata for every other attribute in the same request.

This plugin allows the devices and configurations that have defined expressions to generate their values from those expressions and the reported measure information.

For further information on how the expressions work, refer to the Expression Language Reference.

Allows the devices provisioned in the IoTAgent to map their attributes to more than one entity, declaring the target entity through the Configuration or Device provisioning APIs.

This plugin allows the devices with composite values an expression to update the original values in the devices when the composite expressions are updated in the Context Broker. This behavior is achieved through the use of subscriptions.

IoTAs using this plugins should also define a notification handler to handle incoming values. This handler will be intercepted by the plugin, so the mapped values are included in the updated notification.

When a device is provisioned with bidirectional attributes, the IoTAgent subscribes to changes in that attribute. When a change notification for that attribute arrives to the IoTA, it applies the transformation defined in the device provisioning payload to the notification, and calls the underlying notification handler with the transformed entity.

The following attributes section shows an example of the plugin configuration:

      "attributes": [
        {
          "name":"location",
          "type":"geo:point",
          "expression": "${latitude}, ${longitude}",
          "reverse": [
            {
              "object_id":"longitude",
              "type": "string",
              "expression": "${trim(substr(@location, indexOf(@location, \",\") + 1, length(@location)))}"
            },
            {
              "object_id":"latitude",
              "type": "string",
              "expression": "${trim(substr(@location, 0, indexOf(@location, \",\")))}"
            }
          ]
        }
      ],

For each attribute that would have bidirectionality, a new field reverse must be configured. This field will contain an array of fields that will be created based on the notifications content. The expression notification can contain any attribute of the same entity as the bidirectional attribute; declaring them in the expressions will add them to the subscription payload.

For each attribute in the reverse array, an expression must be defined to calculate its value based on the notification attributes. This value will be passed to the underlying protocol with the object_id name. Details about how the value is then progressed to the device are protocol-specific.

In order to ease the transition from the old IoTAgent implementation (formerly known as IDAS) to the new Node.js based implementations, a data migration tool has been developed. This data migration tool has been integrated as a command in the IoTAgent command line tester.

In order to perform a full migration, follow this steps:

• From the project root, start the command line tester:

bin/iotAgentTester.js

• Configure the MongoDB host and port, and the origin Database (that holds the data to be migrated):

configMigration localhost 27017 originDB

• Launch the migration, using the special value "*" as service and subservice

migrate targetDB * *

Some warnings may appear with the "Attribute [_id] was not found for item translation" message during the migration. They show the values existing in the original DB that had no translation for the target DB.

If you want to restrict the migration for certain services and subservices, just substitute the '*' value for the particular service and subservice you want to use.

A command line client to experiment with the library is packed with it. The command line client can be started using the following command:

bin/agentConsole.js

The client offers an API similar to the one offered by the library: it can start and stop an IoT agent, register and unregister devices, send measures mimicking the device and receive updates of the device data. Take into account that, by default, the console uses the same config.js file than the IoT Agent.

The command line client creates a console that offers the following options:

stressInit

	Start recording a stress batch.

stressCommit <delay> <times> <threads> <initTime>

	Executes the recorded batch as many times as requested, with delay (ms) between commands.
	The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms)
	parameter indicates the mean of the random initial waiting times for each agent.

exit

	Exit from the command line.

start

	Start the IoT Agent

stop

	Stop the IoT Agent

register <id> <type>

	Register a new device in the IoT Agent. The attributes to register will be extracted from the
	type configuration

unregister <id> <type>

	Unregister the selected device

showConfig

	Show the current configuration file

config <newConfig>

	Change the configuration file to a new one

updatevalue <deviceId> <deviceType> <attributes>

	Update a device value in the Context Broker. The attributes should be triads with the following
	format: "name/type/value" sepparated by commas.

listdevices

	List all the devices that have been registered in this IoT Agent session

The library also offers a Context Broker and IoT Agent client that can be used to:

• Simulate operations to the Context Broker used by the IoT Agent, triggering Context Provider forwardings for lazy attributes and checking the appropriate values for active ones.
• Simulate operations to the Device Provisioning API and Configuration API of the IoT Agent.

The tester can be started with the following command, from the root folder of the project:

bin/iotAgentTester.js

From the command line, the help command can be used to show a description of the currently supported features. These are the following:

stressInit

	Start recording a stress batch.

stressCommit <delay> <times> <threads> <initTime>

	Executes the recorded batch as many times as requested, with delay (ms) between commands.
	The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms)
	parameter indicates the mean of the random initial waiting times for each agent.

exit

	Exit from the command line.

update <entity> <type> <attributes>

	Update the values of the defined set of attributes, using the following format: name#type=value(|name#type=value)*

append <entity> <type> <attributes>

	Append a new Entity with the defined set of attributes, using the following format: name:type=value(,name:type=value)*

query <entity> <type>

	Get all the information on the selected object.

queryAttr <entity> <type> <attributes>

	Get information on the selected object for the selected attributes.

discover <entity> <type>

	Get all the context providers for a entity and type.

configCb <host> <port> <service> <subservice>

	Config a new host and port for the remote Context Broker.

showConfigCb

	Show the current configuration of the client for the Context Broker.

configIot <host> <port> <service> <subservice>

	Config a new host and port for the remote IoT Agent.

showConfigIot

	Show the current configuration of the client for the IoT Agent.

provision <filename>

	Provision a new device using the Device Provisioning API. The device configuration is
	read from the file specified in the "filename" parameter.

provisionGroup <template> <data> <type>

	Provision a group of devices with the selected template, taking the information needed to
	fill the template from a CSV with two columns, DEVICE_ID and DEVICE_NAME. The third parameter, type
	will be used to replace the DEVICE_TYPE field in the template. All the devices will be provisioned
	to the same IoT Agent, once the templates have been fulfilled.

listProvisioned

	List all the provisioned devices in an IoT Agent.

removeProvisioned <deviceId>

	Remove the selected provisioned device from the IoT Agent, specified by its Device ID.

addGroup <filename>

	Add a new device group to the specified IoT Agent through the Configuration API. The
	body is taken from the file specified in the "filename" parameter.

listGroups

	List all the device groups created in the selected IoT Agent for the configured service

removeGroup <apiKey> <resource>

	Remove the device group corresponding to the current configured subservice.

authenticate <host> <port> <user> <password> <service>

	Authenticates to the given authentication server, and use the token in subsequent requests.

setProtocol <protocol>

	Sets the protocol to use in the requests (http or https). Defaults to http.

configMigration <host> <port> <originDb>

	Sets the configuration for a migration between a C++ IoTA and a Node.js one.

showConfigMigration

	Shows the current migration configuration.

addProtocols <protocols>

	Add a protocol translation table, in the following format:
		protocolOrigin1=protocolTarget1;protocolOrigin2=protocolTarget2...


migrate <targetDb> <service> <subservice>

	Migrate all the devices and services for the selected service and subservice into the
	specified Mongo database. To perform the migration for all the services or all the
	subservices, use the "*" value.

The agent session stores transient configuration data about the target Context Broker and the target IoT Agent. This configuration is independent, and can be checked with the showConfigCb and showConfigIot commands, respectively. Their values can be changed with the configCb and configIot commands respectively. The new configurations will be deleted upon startup.

The command line testing tools make use of the command-node Node.js library for command line utils. In order to help creating testing tools for IoTAgents of specific protocols, all the commands of the library tester are offered as a array that can be directly imported into other Command Line tools, using the following steps:

• Require the ´iotagent-node-lib´ command line module in your command line tool:

  var iotaCommands = require('iotagent-node-lib').commandLine;

• Initialize the command line utils (the initialization function takes two arguments, that will be explained in detail below:

iotaCommands.init(configCb, configIot);

• Add the IOTA Lib commands to your array of commands

commands = commands.concat(commands, iotaCommands.commands);

• Execute the command line interpreter as usual:

clUtils.initialize(commandLine.commands, 'IoT Agent tester> ');

The command line module makes use of two configuration objects. Both can be shown and edited in the command line using the provided commands, but a default value must be present.

The Context Broker configuration object holds all the information about the Context Broker where the IoT Agent to be tested is connected. It MUST contain the following attributes:

• host: host where the Context Broker instance is located.
• port: port where the Context Broker instance is listening.
• service: service that will be used in all the NGSI operations.
• subservice: service that will be used in all the NGSI operations.

The IoT Agent configuration object holds information about the IoT Agent that is being tested. It MUST contain the following attributes:

• host: host where the IoT Agent instance is located.
• port: port where the IoT Agent instance is listening.
• service: service that will be used to group devices and device information.
• subservice: subservice that will be used to group devices and device information.

Information about developing for the IoTAgent Library can be found at the corresponding section of the User & Programmers Manual.