Code Monkey home page Code Monkey logo

azure-iot-edge-identity-translation-lite's Introduction

page_type languages products
sample
csharp
python
dotnet
azure
azure-iot
azure-iot-edge
azure-iot-hub
azure-functions
vs-code

Azure IoT Edge Identity Translation Lite: Sample on implementing Identity Translation at the edge

This sample builds the required components to support the Idenity Translation pattern with Azure IoT Edge. The Identity Translation Pattern is a pattern by which you have to implement both protocol translation as well as having the devices behind the protocol adopt an identity in IoT Hub. However, the devices themselves don't talk directly to IoT Hub, but rather get their identities impersonated by a custom module in IoT Edge.

This sample is a 'lite' implementation of the Identity Translation pattern as it only supports the basic identity translation flows based on symmetric keys.

Prerequisites

  • An Azure account and access to a subscription where you can provision new resources (IoT Hub, Functions, Storage, Container Registry).
  • Visual Studio Code if you want to run any parts of the sample locally.
  • Docker Desktop for building the container modules (if not using the pre-built images).
  • Azure Powershell az if you are running the automated sample.

Setup

There are two ways of setting up a running demo:

  • Automated sample setup can be found here.
  • There is also an option to setting up everything manually, including building the container images. You can find a step-by-step guide here.

Key concepts

For this Identity Translation Lite sample we are relying on a protocol translation module which is to be seen as something you can replace with your own module. This protocol translation module generates messages into edgeHub with certain headers. These messages can then be picked up by the second module: the Identity Translation module.

The solution also comprises of a cloud section that takes care of provisioning the device on the IoT Hub and assigning it as a child of the IoT Edge device. To prevent any type of child device to be provisioned, there is a whitelisting file that validates whether the child device can be provisioned. This cloud solution leverages an Azure Function that gets triggered by an Event Grid subscription.

Registration flow diagram

Sample setup flow

The sample setup flow uses an unsecure MQTT broker connection (mosquitto) to interact with clients (represented by the Python script sim_clients.py) and a custom implemented protocol translation module (ptm-mqtt) to interact with the identity translation module.

Sample flow diagram

Open points

This is a sample implementation of the Identity Translation pattern. The following functionalities are not implemented.

  • DirectMethods to leaf devices
  • DeviceTwins for leaf devices
  • Persistance of leaf device registrations (the registration flow is initiated automatically when a leaf device starts sending messages)
  • Disable or delete leaf devices in IoTHub
  • The deployment ARM template refers to an external Git repo for Function deployment. Change once this repo is public.
  • Automate the Python install and sample code run

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

azure-iot-edge-identity-translation-lite's People

Contributors

amit12cool avatar asdamp avatar gewarren avatar guybartal avatar katriendg avatar michielvanschaik avatar microsoft-github-operations[bot] avatar microsoftopensource avatar myamama avatar vishnureddy17 avatar vslepakov avatar williamberryiii avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

azure-iot-edge-identity-translation-lite's Issues

issue when start module mosquitto

I got this error when following your doocument.

Here is the log from mosquitto module:

617728741: Config loaded from /mosquitto/config/mosquitto.conf.
1617728741: Starting in local only mode. Connections will only be possible from clients running on this machine.
1617728741: Create a configuration file which defines a listener to allow remote access.
1617728741: For more details see https://mosquitto.org/documentation/authentication-methods/
1617728741: Opening ipv4 listen socket on port 1883.
1617728741: Opening ipv6 listen socket on port 1883.
1617728741: Error: Address not available
1617728741: mosquitto version 2.0.10 running

Note this line:
1617728741: Error: Address not available
this error result in I can't connect mqtt server when I excute your python code in order to creating client.

here is the log when exicuting python script:
06-Apr-21 12:19:46.475 - INFO - [sim_clients.py:] creating client with client_id = client3
06-Apr-21 12:19:46.475 - INFO - [sim_clients.py:] creating client with client_id = client4
06-Apr-21 12:19:46.475 - INFO - [sim_clients.py:] creating client with client_id = client5
06-Apr-21 12:19:46.480 - ERROR - [sim_clients.py:] client0 is not connected to the broker!
06-Apr-21 12:19:46.480 - ERROR - [sim_clients.py:] client1 is not connected to the broker!
06-Apr-21 12:19:46.480 - ERROR - [sim_clients.py:] client2 is not connected to the broker!
06-Apr-21 12:19:46.480 - ERROR - [sim_clients.py:] client3 is not connected to the broker!
06-Apr-21 12:19:46.480 - ERROR - [sim_clients.py:] client4 is not connected to the broker!
06-Apr-21 12:19:46.480 - ERROR - [sim_clients.py:] client5 is not connected to the broker!
06-Apr-21 12:19:47.482 - ERROR - [sim_clients.py:] client0 is not connected to the broker!

Unable to open Events link as the connection could not be authenticated

Setup

  1. Installed IoT Edge Runtime in Linux container on windows 10 machine
  2. Registered IoT edge in IoT Hub and created few leaf devices as child devices under this parent device

Steps to Reproduce

  1. Created Signed Key using SignAsync Method
  2. Created DeviceClient using signed key, Iot Hub hostname & gateway name for one of the leaf device
  3. On Sending the message to using DeviceClient, getting below error.

Stack Trace:

Unhandled Exception: System.AggregateException: One or more errors occurred. (error(condition:amqp:not-allowed,description:Unable to open Events link as the connection could not be authenticated)) ---> System.InvalidOperationException: error(condition:amqp:not-allowed,description:Unable to open Events link as the connection could not be authenticated) ---> Microsoft.Azure.Amqp.AmqpException: Unable to open Events link as the connection could not be authenticated

at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)

at Microsoft.Azure.Amqp.AmqpObject.OpenAsyncResult.End(IAsyncResult result)

at Microsoft.Azure.Amqp.AmqpObject.EndOpen(IAsyncResult result)

at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)

--- End of stack trace from previous location where exception was thrown ---

at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSession.OpenSendingAmqpLinkAsync(DeviceIdentity deviceIdentity, AmqpSession amqpSession, Nullable1 senderSettleMode, Nullable1 receiverSettleMode, String deviceTemplate, String moduleTemplate, String linkSuffix, String CorrelationId, TimeSpan timeout)

--- End of inner exception stack trace ---

at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSession.OpenSendingAmqpLinkAsync(DeviceIdentity deviceIdentity, AmqpSession amqpSession, Nullable1 senderSettleMode, Nullable1 receiverSettleMode, String deviceTemplate, String moduleTemplate, String linkSuffix, String CorrelationId, TimeSpan timeout)

at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSession.OpenTelemetrySenderLinkAsync(DeviceIdentity deviceIdentity, TimeSpan timeout)

at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout)

at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout)

at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken)

at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken)

at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation)

at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken)

at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass14_0.<b__0>d.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.SendEventAsync(Message message, CancellationToken cancellationToken)

at Microsoft.Azure.Devices.Client.InternalClient.SendEventAsync(Message message)

at IdentityTranslationLite.Program.PushMessages() in /app/Program.cs:line 107

at IdentityTranslationLite.Program.Init() in /app/Program.cs:line 84

--- End of inner exception stack trace ---

at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)

at System.Threading.Tasks.Task.Wait()

at IdentityTranslationLite.Program.Main(String[] args) in /app/Program.cs:line 42

Can this be used in production level?

Hi,

We want to use Azure IoT Edge as an identity translation gateway with the protocol translation gateway. Because we are working with an OBD-II device that strictly uses TCP/IP as communication protocol. We can only add IP, PORT, and APN in the device nothing else.

We want to use these devices with the Azure IoT Hub. As we know Azure IoT hub works only with MQTT, AMQP, and HTTPS.

We want to use Identity translation for:

  1. Creating IDs for these lots of devices from the IoT edge and register them in IoT Hub.
  2. If possible, we want to automatically make the device to Azure IoT Hub communication via IoT Edge.

We want to use protocol translation for:

  1. Acting as a medium to communicate between the Devices working on the TCP/IP and the Azure IoT Hub, which supports MQTT, AMQP, and HTTPS communication protocol.

We want to have a Bi-directional communication with the Azure IoT cloud. So we can send messages back to the devices when needed and do FOTA.

My Questions:

  1. Is there a module available for the TCP/IP implementation?
  2. Does this repository can be used for production-level implementation for the above?
  3. Is there any full detailed documentation like this sample for implementing the actual asked for production level?
  4. Does this Implementation way provides two-way communication? Device to edge to cloud and then Cloud to edge to the device?
  5. Do we have to create manually each device in the gateway as the real devices and do manual registration for each of them?
    This means If I have 1000 devices in real-time, do I have to create 1000 virtual devices in the gateways manually and then register them manually on the IoT Hub?
  6. Is this identity translation gateway in development and is actually not fully blown for the production purpose?
  7. Is there any working example or a full developer guide for Implementing Azure IoT Edge gateway as an Identity translation gateway with protocol translation gateway?

Facing issue in deploying azure function

I'm facing an issue on publish step in VSCode for Azure function. The error log is below:-

/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : It was not possible to find any compatible framework version [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : The framework 'Microsoft.NETCore.App', version '2.1.0' was not found. [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :   - The following frameworks were found: [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :       3.1.4 at [/usr/share/dotnet/shared/Microsoft.NETCore.App] [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :  [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : You can resolve the problem by installing the specified framework and/or SDK. [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :  [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : The specified framework can be found at: [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :   - https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=2.1.0&arch=x64&rid=ubuntu.18.04-x64 [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error :  [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]
/home/amit/.nuget/packages/microsoft.net.sdk.functions/1.0.29/build/netstandard1.0/Microsoft.NET.Sdk.Functions.Build.targets(41,5): error : Metadata generation failed. [/home/amit/Desktop/Projects/Barco/azure-iot-edge-identity-translation-lite/src/cloud/functions/LeafDeviceIoTHubRegistration.csproj]

Im on an Ubuntu 18.04 machine with dotnet --version is 3.1.300

facing issue with ptm-mqtt module

ConnectOperation: completing with error None caused by [Errno -2] Name or service not known
RetryStage(ConnectOperation): Op needs retry with interval 20 because of None caused by [Errno -2] Name or service not known. Setting timer.
transport.connect raised error
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/azure/iot/device/common/mqtt_transport.py", line 340, in connect
host=self._hostname, port=8883, keepalive=DEFAULT_KEEPALIVE
File "/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py", line 937, in connect
return self.reconnect()
File "/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py", line 1071, in reconnect
sock = self._create_socket_connection()
File "/usr/local/lib/python3.7/site-packages/paho/mqtt/client.py", line 3522, in _create_socket_connection
return socket.create_connection(addr, source_address=source, timeout=self._keepalive)
File "/usr/local/lib/python3.7/socket.py", line 707, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/local/lib/python3.7/socket.py", line 752, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/azure/iot/device/common/pipeline/pipeline_stages_mqtt.py", line 115, in _run_op
self.transport.connect(password=self.sas_token)
File "/usr/local/lib/python3.7/site-packages/azure/iot/device/common/mqtt_transport.py", line 345, in connect
raise exceptions.ConnectionFailedError(cause=e)
azure.iot.device.common.transport_exceptions.ConnectionFailedError: None caused by [Errno -2] Name or service not known

Question about downstream device identity creation

Hi, in this sample, the IoT identity for the downstream device is created in an Azure Function. Is there a reason for this? Why is this not done in the Edge module itself, wouldn't it be easier?

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.