Comments (10)
Hey @egil, glad to see you have the service starting in the container. To answer the question about EdgeAgent I would need to take a look at EdgeAgent logs. Double check that you have a deployment that targets the device. EdgeHub will not start if there has been no deployment applied to the device.
Another possibility is that EdgeAgent can't connect to the IoT Hub. This is unlikely, because connectivity checks have passed when you ran check
command. But if it is indeed connectivity problems - you'd need to look into that.
from iotedge.
Guys, I have a proof of concept working as of today (maybe kind of a more edgy approach though)... however, it works for testing on my Mac M3 Pro
Relevant additions in Dockerfile
- using
arm64v8/ubuntu:22.04
- install docker-ce (and add-ons) inside the container directly through
Dockerfile
FROM arm64v8/ubuntu:22.04
RUN apt-get -y update && \
apt-get remove docker docker.io containerd runc && \
apt-get -y install --no-install-recommends iputils-ping curl net-tools software-properties-common iproute2
RUN add-apt-repository "deb [arch=arm64] https://download.docker.com/linux/ubuntu jammy stable"; exit 0
RUN apt-get -y update --allow-insecure-repositories
RUN apt-get -y install --allow-unauthenticated --no-install-recommends \
docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
(yes, I know this is a hacky way to install)
- Remove the line installing
moby-cli
- Removed the
VOLUME
andEXPOSE
statements
Configuration of Azure Iot-Hub
- created a deployment in Azure IoT-Hub targeting my device, something like
{
"modulesContent": {
"$edgeAgent": {
"properties.desired": {
"modules": {
"redis": {
"settings": {
"image": "redis:7-alpine",
"createOptions": "{\"Cmd\": [\"redis-server\", }"
},
"type": "docker",
"status": "running",
"restartPolicy": "always",
"version": "1.0",
"startupOrder": 1
},
"docker-autoheal": {
"settings": {
"image": "docker.io/willfarrell/autoheal:1.2.0",
"createOptions": "{\"HostConfig\": {\"Binds\": [\"/var/run/docker.sock:/var/run/docker.sock\"]}}"
},
"type": "docker",
"status": "running",
"restartPolicy": "always",
"version": "1.0",
"startupOrder": 3
}
},
"runtime": {
"settings": {
"minDockerVersion": "v1.25",
"registryCredentials": {
"<containerregistryname>": {
"address": "<something>",
"username": "<user>",
"password": "<pass>"
}
}
},
"type": "docker"
},
"schemaVersion": "1.1",
"systemModules": {
"edgeAgent": {
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.4.9",
"createOptions": "{\"HostConfig\": {\"Binds\": [\"/persist/edgeAgent:/persist/edgeAgent\"]}}"
},
"type": "docker",
"env": {
"storageFolder": {
"value": "/persist/edgeAgent"
},
"SendRuntimeQualityTelemetry": {
"value": "false"
},
"RocksDB_MaxTotalWalSize": {
"value": 2097152
},
"RocksDB_MaxManifestFileSize": {
"value": 2097152
},
"Storage_LogLevel": {
"value": "ERROR"
}
}
},
"edgeHub": {
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.4.9",
"createOptions": "{\"HostConfig\": {\"PortBindings\": {\"443/tcp\": [{\"HostPort\": \"443\"}], \"5671/tcp\": [{\"HostPort\": \"5671\"}],\"8883/tcp\": [{\"HostPort\": \"8883\"}]},\"Binds\": [\"/persist/edgeHub:/persist/edgeHub\"]}}"
},
"type": "docker",
"status": "running",
"restartPolicy": "always",
"startupOrder": 0,
"env": {
"OptimizeForPerformance": {
"value": "false"
},
"amqpSettings__enabled": {
"value": true
},
"AmqpSettings__Enabled": {
"value": true
},
"httpSettings__enabled": {
"value": true
},
"HttpSettings__Enabled": {
"value": true
},
"mqttSettings__enabled": {
"value": true
},
"MqttSettings__Enabled": {
"value": true
},
"RocksDB_MaxTotalWalSize": {
"value": 2097152
},
"RocksDB_MaxManifestFileSize": {
"value": 2097152
},
"Storage_LogLevel": {
"value": "ERROR"
},
"storageFolder": {
"value": "/persist/edgeHub"
},
"CloudOperationTimeoutSecs": {
"value": "120"
}
}
}
}
}
},
"$edgeHub": {
"properties.desired": {
"routes": {
"route": "FROM /messages/* INTO $upstream"
},
"schemaVersion": "1.1",
"storeAndForwardConfiguration": {
"timeToLiveSecs": 172800
}
}
}
}
}
To be deployed with something like
az iot edge deployment create \
--hub-name "$IOTHUB_NAME" \
--target-condition "$TARGET_CONDITION" \
--deployment-id "deployment" \
--content "<JSON above>"
- Running everything with
docker run -it --rm \
--privileged \
--hostname $DEVICE_NAME \
--name=$DEVICE_NAME \
--dns 8.8.8.8 \
--log-driver "json-file" \
--log-opt "max-file=10" \
--log-opt "max-size=200k" \
iot-edge-device-docker
(note: no -v /var/run/docker.sock:/var/run/docker.sock -v /sys/fs/cgroup:/sys/fs/cgroup:rw
here!)
More comments
- non-privileged
docker run
should be possible withsysbox-runc
runtime https://github.com/nestybox/sysbox/releases/tag/v0.6.2 (however, they don't provide a package for Mac) - provisioning using DPS (device provisioning service) works also, toml file must be adapted then
Thank you very much for this issue. Your snippets above were super useful!! ❤️
Any feedback welcome!
Cheers, Thomas
from iotedge.
@egil remember to put your container registry settings into the json :-)
- you can see some output in
docker logs -f --tail 500 <your-build>
on the system running the stack - you can do
docker exec -it <your-build> bash
and then, inside the containerjournalctl -xef
HTH, THomas
from iotedge.
Hey @egil, this is not a use case that we can support. IoT Edge runs as a set of host services, and needs systemd to run. And most likely you won't be able to achieve this, especially if your host does not have systemd. You can try to run the container on Ubuntu or to install systemd within the container itself with dockerfile, but IDK if that will work. The simplest way is to create a VM image instead.
I quickly searched online and found something similar - https://askubuntu.com/questions/1471990/add-systemd-to-boot-of-a-docker-container
from iotedge.
Apparently, there was an old issue about that as well - so take a look - #7161
from iotedge.
Hey @vadim-kovalyov, yes, I know it's not directly supported. I've seen other examples that did this for old versions of the edge device software, and its certainly preferred to having to run a VM in the cloud or locally.
I did manage to get it running and the edge device is able to connect to my IoT Hub, but only the agent is getting deployed. There are some warnings if I do an iotedge check
. If you don't mind taking a look and let me know if these are deal breakers:
root@cb7a0509e394:/# iotedge check --verbose
Configuration checks (aziot-identity-service)
---------------------------------------------
√ keyd configuration is well-formed - OK
√ certd configuration is well-formed - OK
√ tpmd configuration is well-formed - OK
√ identityd configuration is well-formed - OK
√ daemon configurations up-to-date with config.toml - OK
√ identityd config toml file specifies a valid hostname - OK
√ aziot-identity-service package is up-to-date - OK
√ host time is close to reference time - OK
√ preloaded certificates are valid - OK
√ keyd is running - OK
√ certd is running - OK
√ identityd is running - OK
√ read all preloaded certificates from the Certificates Service - OK
√ read all preloaded key pairs from the Keys Service - OK
√ check all EST server URLs utilize HTTPS - OK
√ ensure all preloaded certificates match preloaded private keys with the same ID - OK
Connectivity checks (aziot-identity-service)
--------------------------------------------
√ host can connect to and perform TLS handshake with iothub AMQP port - OK
√ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with iothub MQTT port - OK
Configuration checks
--------------------
√ aziot-edged configuration is well-formed - OK
√ configuration up-to-date with config.toml - OK
√ container engine is installed and functional - OK
√ configuration has correct URIs for daemon mgmt endpoint - OK
√ aziot-edge package is up-to-date - OK
√ container time is close to host time - OK
‼ DNS server - Warning
Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
You can ignore this warning if you are setting DNS server per module in the Edge deployment.
caused by: Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
You can ignore this warning if you are setting DNS server per module in the Edge deployment.
‼ production readiness: logs policy - Warning
Container engine is not configured to rotate module logs which may cause it run out of disk space.
Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
You can ignore this warning if you are setting log policy per module in the Edge deployment.
caused by: Container engine is not configured to rotate module logs which may cause it run out of disk space.
Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
You can ignore this warning if you are setting log policy per module in the Edge deployment.
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
Data might be lost if the module is deleted or updated.
Please see https://aka.ms/iotedge-storage-host for best practices.
caused by: The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
Data might be lost if the module is deleted or updated.
Please see https://aka.ms/iotedge-storage-host for best practices.
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
Could not check current state of edgeHub container
caused by: Could not check current state of edgeHub container
caused by: docker returned exit status: 1, stderr = Error: No such object: edgeHub
√ Agent image is valid and can be pulled from upstream - OK
√ proxy settings are consistent in aziot-edged, aziot-identityd, moby daemon and config.toml - OK
Connectivity checks
-------------------
√ container on the default network can connect to upstream AMQP port - OK
√ container on the default network can connect to upstream HTTPS / WebSockets port - OK
√ container on the default network can connect to upstream MQTT port - OK
skipping because of not required in this configuration
√ container on the IoT Edge module network can connect to upstream AMQP port - OK
√ container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - OK
√ container on the IoT Edge module network can connect to upstream MQTT port - OK
skipping because of not required in this configuration
31 check(s) succeeded.
3 check(s) raised warnings.
1 check(s) raised errors.
2 check(s) were skipped due to errors from other checks.
Here is the command used to build and run the docker container:
docker build -t iot-edge-device-docker .
docker run --privileged -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /sys/fs/cgroup:/sys/fs/cgroup:rw -v C:\temp\iotedge:/iotedge/storage -e connectionString='<INSERT EDGE DEVICE CONNECTION STRING>' --name iot-edge-device iot-edge-device-docker --hostname=egh-edge-device --dns 8.8.8.8 --dns 8.8.4.4
Here are the files used to set up the docker container:
Dockerfile
# Start from Ubuntu for `apt-get`
FROM ubuntu:22.04
RUN apt-get update -qq && apt-get install -qqy \
apt-transport-https \
ca-certificates \
curl \
wget \
gnupg \
lsb-release \
jq \
net-tools \
iptables \
iproute2 \
systemd && \
rm -rf /var/lib/apt/lists/*
# Cleanup to enable systemd and systemctl commands
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
# Step 1: Install Microsoft package repository
RUN wget https://packages.microsoft.com/config/ubuntu/22.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb && \
dpkg -i packages-microsoft-prod.deb && \
rm packages-microsoft-prod.deb
# Step 2: Install Moby engine and Azure CLI
RUN apt-get update && apt-get install -y moby-cli moby-engine
# Step 3: Configure Docker daemon
RUN echo '{ "log-driver": "local" }' > /etc/docker/daemon.json
# Step 4: Install Azure IoT Edge
RUN apt-get install -y aziot-edge
# Clean up to reduce image size
RUN apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Allow services to start
VOLUME [ "/sys/fs/cgroup", "/var/lib/docker" ]
EXPOSE 2375
EXPOSE 15580
EXPOSE 15581
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
COPY iotedge-init.service /etc/systemd/system/iotedge-init.service
COPY edge-init.sh /usr/local/bin/edge-init.sh
RUN chmod +x /usr/local/bin/edge-init.sh
RUN systemctl enable iotedge-init.service
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
docker-entrypoint.sh
#!/bin/bash
echo "***Configuring IoT Edge Runtime***"
cat <<EOF > /etc/aziot/config.toml
[provisioning]
source = "manual"
connection_string = "$connectionString"
[agent]
name = "edgeAgent"
type = "docker"
[agent.config]
image = "mcr.microsoft.com/azureiotedge-agent:1.4"
createOptions = { HostConfig = { Binds = ["/iotedge/storage:/iotedge/storage"] } }
[connect]
workload_uri = "unix:///var/run/iotedge/workload.sock"
management_uri = "unix:///var/run/iotedge/mgmt.sock"
[listen]
workload_uri = "fd://aziot-edged.workload.socket"
management_uri = "fd://aziot-edged.mgmt.socket"
[moby_runtime]
uri = "unix:///var/run/docker.sock"
network = "azure-iot-edge"
EOF
mkdir -p /iotedge/storage
echo "***Starting systemd***"
exec /lib/systemd/systemd --log-level=info
iotedge-init.service
[Unit]
Description=Initialize IoT Edge Runtime
After=docker.service
Requires=docker.service
[Service]
Type=oneshot
ExecStart=/usr/local/bin/edge-init.sh
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
edge-init.sh
#!/bin/bash
echo "***Starting Docker in Docker***"
# Updated approach to start Docker
# Check for the presence of docker daemon
pgrep dockerd
if [ $? -ne 0 ]; then
dockerd &
else
echo "Docker daemon already running."
fi
# Wait for Docker to initialize
while (! docker stats --no-stream ); do
echo "Waiting for Docker to launch..."
sleep 1
done
iotedge config apply -c /etc/aziot/config.toml
from iotedge.
Awesome @muelleth. I am using Docker for Windows, will this also work for me? Do prefer to not have privilege mode enabled (I am a docker newbie).
@vadim-kovalyov thanks, I am also an iot hub newbie, so I am probably just missing the "edge deployment" @muelleth described above, so that is probably why the edgeClient is not getting pushed to my docker container.
from iotedge.
@vadim-kovalyov I tried creating an deployment using @muelleth deployment file as a template and assigned it to my edge device. It failed to deploy the three images though. Can you guide me to where I need to look for logs you need to help me debug?
Appreciate all the help with this!
from iotedge.
It did indeed help. Got it running with my dockerfile above. Thanks. Now to develop a few modules 😊
from iotedge.
I'll close this issue and leave folks with a link to my docker image:
- https://hub.docker.com/r/egilhansen/iothub-edge-device
- https://github.com/egil/IoT-Edge-Device-in-Docker
from iotedge.
Related Issues (20)
- IoT device works perfectly for days, then it does not - reboot fixes issue HOT 3
- Error getting device scope result from IoTHub (IotHubUnauthorizedAccess) HOT 2
- Unexpected shutdown, is it a kind of planned upgrade? HOT 4
- Direct Method of IoTEdge module takes 3 minutes to start responding HOT 11
- IoT Edge produces too much traffic with default configuration, it is not conducive for billable networks such as cellular. HOT 13
- Iotedge fails to make docker-proxy.sock after reboot (file exists) in Ubuntu Core HOT 13
- EdgeAgent does not restart crashed module if busy downloading new manifest 1.4.10 HOT 3
- Create a reusable Volume in Azure IOT Edge Module via Container Create Options HOT 2
- Can not ssh into a (running) VM with Ubuntu Core 20 on my k3s cluster HOT 1
- Reported twin does not reflect running container HOT 2
- Scarthgap support? HOT 1
- Best Practices/Guidance on upgrading a remote database module HOT 3
- edgeAgent hangs after a networking issue. HOT 2
- [Documentation] [Low priority] Add to the config.toml template and documentation a section for logging HOT 2
- "Did not receive ack for message / System.TimeoutException: Message completion response not received" errors after environment upgrade HOT 3
- Iot edge not starting on debian bullseye HOT 2
- IoT Edge child node unable to connect to IoT Edge transparent gateway
- User module unable to connect to Hub with 'Communication_error'
- EdgeHub fails to parse valid D2C creation time UTC string
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from iotedge.