aws-greengrass / aws-greengrass-docker Goto Github PK

Describe the bug
is it possible to use greengrass-cli inside docker container which was created from aws-greengrass/aws-greengrass-docker repositry?

https://docs.aws.amazon.com/greengrass/v2/developerguide/gg-cli-deployment.html

can you provide CLI examples of such usage?

Describe the bug
Current release only supports AWS IoT Greengrass 2.1.0, the latest released version of Greengrass is 2.2.0

To Reproduce
Download main branch and build.

Expected behavior
I download from main branch and build. At that point I have a container file for 2.2.0.

Actual behavior
Greengrass 2.1.0 is build and used.

Screenshots
n/a

Docker Inspect and Logs
n/a

Environment

• OS:
• JDK version:
• Greengrass version
• Docker version
• Docker image version
• docker-compose version
• Output of uname -a
• Output of cat /etc/os-release or its equivalent

Additional context

• An authoritative has file for the Greengrass 2.2.0.zip file is needed
• A complete series of tests should be performed before release

Describe the bug
In order to run components as other users, Greengrass needs specific sudo permission. The greengrass-entrypoint.sh script modified the /etc/sudoers file to add the right permission, but this is only performed the first time the container is started on a empty GGC_ROOT folder.

To Reproduce
Run the container once. Check the /etc/sudoers file -> it is correct.
Stop the container. Start the container again with the same mounted volumes (/etc is not mounted externally, and it is not possible to do so)
Check the /etc/sudoers file -> it contains root ALL=(ALL) ALL

Expected behavior
/etc/sudoers should have the right content upon container restarts

Actual behavior
/etc/sudoers has wrong content upon container restart

Screenshots
If applicable, add screenshots to help explain your problem.

Docker Inspect and Logs
If available, add docker_container.log and docker_inspect.json.

Environment

• OS: [e.g. Ubuntu 20.04]
• JDK version:
• Greengrass version
• Docker version
• Docker image version
• docker-compose version
• Output of uname -a
• Output of cat /etc/os-release or its equivalent

Additional context
Add any other context about the problem here.

Can we add support for Python3.9?

Describe the bug
We created our custom provisioning plugins
https://docs.aws.amazon.com/greengrass/v2/developerguide/develop-custom-provisioning-plugins.html
and use it via TRUSTED_PLUGIN key during greengrass container start

We exposed port for plugin invoke and discovered that it was exposed on IPv6 addressing.
Output of netstat inside greengrass container:

bash-4.2# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp6 0 0 127.0.0.1:8090 :::* LISTEN 58/java

Expected behavior
IPv4 addresing

Actual behavior
IPv6 addressing

Additional context
Can it be controlled via greengrass code?
Do you have some options?

Describe the bug
Running the container as explained in the README prints the following error and provisioning cannot be done:

/greengrass-entrypoint.sh: 19: [: true: unexpected operator

To Reproduce
Steps to reproduce the behavior. If possible, provide a minimal amount of code that causes the bug.

Running the container as explained in the README, passing -e PROVISION=true to docker run

Expected behavior

Should be working. The fix is simple, replace == with = since == is not supported by all shells and is not portable (see https://www.shellscript.sh/test.html)

Actual behavior

Does not work

Screenshots
If applicable, add screenshots to help explain your problem.

Docker Inspect and Logs
If available, add docker_container.log and docker_inspect.json.

Environment

• OS: [e.g. Ubuntu 20.04]
• JDK version:
• Greengrass version
• Docker version
• Docker image version
• docker-compose version
• Output of uname -a
• Output of cat /etc/os-release or its equivalent

Additional context
Add any other context about the problem here.

v2.6.0 was released recently. We should update this project now.

https://docs.aws.amazon.com/greengrass/v2/developerguide/greengrass-release-2022-06-27.html

Feature Description
Provisioning Greengrass V2 locally without docker is possible when the AWS credentials (Access key ID and Secret Access Key) as well as the configured default region are exposed via environment variables.
This is useful when working with tools like aws-vault, which expose AWS credentials and region via environment variables.

Use Case
Be able to retrieve AWS credentials from a secret store and configured default region via aws-vault and provision Greengrass V2 in docker using temporary credentials.

Proposed Solution
Introduce new environment variables:

• AWS_REGION
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_SESSION_TOKEN

In addition to retrieving credentials only from a mounted config-file, also check if there are environment variables (AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN) exposed and use those.

The priority to use either the config file or environment variables first for provisioning, doesn't matter for me.

Make sure to unset the secret environment variables when provisioning is finished for security reasons (same behavior as when provisioning locally aws-vault exec <PROFILENAME> -- java -jar ... Greengrass.jar).

When running the container, the docker run- or the docker-compose up-command need to incorporate the variables:

docker run -e “AWS_REGION=${AWS_REGION}” -e "AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}" -e "AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}" -e "AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN}"...

or

services:
  greengrass:
    environment:
      - AWS_REGION=$AWS_REGION
      - AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
      - AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
      - AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN

The docker run- or docker-compose up-command then needs to be wrapped with aws-vault:

aws-vault exec <PROFILE> -- docker run ...
aws-vault exec <PROFILE> -- docker-compose up

Other

• aws-vault, Installation and Quick Start in README
• How aws-vault works
• useful command during development: aws-vault exec <PROFILENAME> -- env | grep AWS_
• code location to change

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

Describe the bug
Components didn't deployed with use of INIT_CONFIG file

To Reproduce

1. Docker image was build from Dockerfile in this repo GREENGRASS_RELEASE_VERSION=2.10.3

2. Docker run command:

docker run --rm --init -d --name greengrass-test \
 -v /home/user/IOT/config:/tmp/config/:ro \
 -e INIT_CONFIG=/tmp/config/config.yaml \
 --env-file .env \
 -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
 -p 8883 greengrass:3

3. .env file

AWS_REGION=eu-central-1
PROVISION=true
COMPONENT_DEFAULT_USER=ggc_user:ggc_group
INIT_CONFIG=config.yaml
TES_ROLE_ALIAS_NAME=TokenExchangeRoleAlias
DEPLOY_DEV_TOOLS=true
THING_NAME=test-thing
THING_GROUP_NAME=thing-group1
TES_ROLE_NAME=TokenExchangeRole
LOG_LEVEL=INFO

4. Content of config file:

  aws.greengrass.Cli:
    componentType: "PLUGIN"
    configuration:
      AuthorizedPosixGroups: null
      AuthorizedWindowsGroups: null
    dependencies:
    - "aws.greengrass.Nucleus:SOFT"
    version: "2.11.0"
  aws.greengrass.clientdevices.Auth:
    componentType: "PLUGIN"
    configuration:
      ca_type: null
      certificateAuthority:
        caType: null
      deviceGroups:
        definitions:
          AllPermissiveDeviceGroup:
            policyName: "AllPermissivePolicy"
            selectionRule: "thingName: *"
        formatVersion: "2021-03-05"
        policies:
          AllPermissivePolicy:
            AllowAll:
              operations:
              - "*"
              resources:
              - "*"
              statementDescription: "Allow client devices to perform all actions."
      metrics: {}
      security: {}
      startupTimeoutSeconds: "120"
    version: "2.4.3"
  aws.greengrass.clientdevices.mqtt.Bridge:
    componentType: "PLUGIN"
    configuration:
      mqttTopicMapping:
        CloudDeleteShadowNameMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/shadow/name/+/delete/+"
        CloudGetJobIdsMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/jobs/+/get/+"
        CloudGetJobsMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/jobs/get/+"
        CloudGetShadowNameMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/shadow/name/+/get/+"
        CloudNotifyJobsMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/jobs/notify"
        CloudNotifyNextJobsMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/jobs/notify-next"
        CloudUpdateJobsMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/jobs/+/update/+"
        CloudUpdateShadowNameMapping:
          source: "IotCore"
          target: "LocalMqtt"
          topic: "$aws/things/+/shadow/name/+/update/+"
        DeleteShadowNameMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/shadow/name/+/delete"
        EventsMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/rules/events_bi_rule/+"
        FaultsMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/rules/faults_bi_rule/+"
        GetJobIdsMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/jobs/+/get"
        GetJobsMappingMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/jobs/get"
        GetShadowNameMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/shadow/name/+/get"
        MetricsMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/rules/metrics_bi_rule/+"
        PropertiesMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/rules/properties_bi_rule/+"
        StartNextJobsMappingMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/jobs/start-next"
        TelemetryNameMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "dt/iotagent/#"
        UpdateJobsMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/jobs/+/update"
        UpdateShadowNameMapping:
          source: "LocalMqtt"
          target: "IotCore"
          topic: "$aws/things/+/shadow/name/+/update"
      startupTimeoutSeconds: "120"
    dependencies:
    - "aws.greengrass.clientdevices.Auth:HARD"
    lifecycle:
      startup:
        timeout: "120"
    version: "2.2.6"
  aws.greengrass.clientdevices.mqtt.Moquette:
    componentType: "PLUGIN"
    configuration:
      moquette:
        host: "0.0.0.0"
        netty.mqtt.message_size: "131072"
        ssl_port: "8883"
      netty.channel.read.limit: 524288
      netty.channel.write.limit: 524288
      startupTimeoutSeconds: "120"
    dependencies:
    - "aws.greengrass.clientdevices.Auth:HARD"
    lifecycle:
      startup:
        timeout: "120"
    version: "2.3.3"
  aws.greengrass.LocalDebugConsole:
    componentType: "PLUGIN"
    configuration:
      bindHostname: "0.0.0.0"
      httpsEnabled: "false"
      port: "1441"
      websocketPort: "1442"
    dependencies:
    - "aws.greengrass.Cli:SOFT"
    - "aws.greengrass.Nucleus:SOFT"
    lifecycle: {}
    version: "2.3.1"
  aws.greengrass.Nucleus:
    componentType: "NUCLEUS"
    configuration:
      awsRegion: "eu-central-1"
      componentStoreMaxSizeBytes: "10000000000"
      deploymentPollingFrequencySeconds: "15"
      envStage: "prod"
      fleetStatus:
        periodicStatusPublishIntervalSeconds: 86400
      greengrassDataPlaneEndpoint: ""
      greengrassDataPlanePort: "8443"
      mqtt:
        spooler: {}
      networkProxy:
        proxy: {}
      platformOverride: {}
      runWithDefault:
        posixUser: "ggc_user:ggc_group"
      telemetry: {}
    previousVersion: "2.11.0"
    version: "2.11.0"

Expected behavior
Some components files appear in /greengrass/v2/bin and /greengrass/v2/packages folders

Actual behavior
Folders /greengrass/v2/bin and /greengrass/v2/packages(with subfolders) are empty
Auto-provisioning works and services: part of effectiveConfig.yaml a same as config.yaml

Feature Description
Standardize licensing on Apache 2.0

Use Case
Simplifying corporate approval to use AWS Greengrass

Proposed Solution
Either switch the license for aws-greengrass-docker from the MIT license to Apache 2.0 or dual license the component

Other

• ⚠️ This feature might incur a breaking change

Feature Description
Provide instructions to build the container image for aarch64

Use Case
To run this container on ARM machines, and on Mac M1 without the need of an emulator

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

Instructions under item 1.2.1 of the README state the following modifications to docker-compose.yml:

environment:  
 - PROVISION=truevolumes:
 - /path/to/credential/directory/:/root/.aws/:ro

There seems to be a missing line break after true.

Describe the bug

greengrass-cli can't start inside docker container

user@host:# docker exec -it greengrass-cli
OCI runtime exec failed: exec failed: unable to start container process: exec: "greengrass-cli": executable file not found in $PATH: unknown

according to documentation https://docs.aws.amazon.com/greengrass/v2/developerguide/install-gg-cli.html
DEPLOY_DEV_TOOLS=true
PROVISION=true
should be enough for greengrass-cli installation but it wasn't installed

To Reproduce

Docker run command:

docker run --rm --init -d --name greengrass-test \
 -v /home/user/IOT/config:/tmp/config/:ro \
 --env-file .env \
 -e AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY -e AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN \
 -p 8883 greengrass:3

Content of .env file:

GGC_ROOT_PATH=/greengrass/v2
AWS_REGION=eu-central-1
PROVISION=true
COMPONENT_DEFAULT_USER=ggc_user:ggc_group
TES_ROLE_ALIAS_NAME=TokenExchangeRoleAlias
DEPLOY_DEV_TOOLS=true
THING_NAME=test-thing
THING_GROUP_NAME=thing-group1
TES_ROLE_NAME=TokenExchangeRole
LOG_LEVEL=DEBUG

Expected behavior
Run commands via
docker exec -it greengrass-cli

Docker Inspect and Logs
No new records in /greengrass/v2/logs/greengrass.log

Environment
GREENGRASS_RELEASE_VERSION=2.10.3

Upgrade to 2.8.0 https://github.com/aws-greengrass/aws-greengrass-nucleus/releases/tag/v2.8.0

Describe the bug
Python lambdas work just fine. However whenever I'm using a nodejs lambda I get the following error.

`2022-06-23T02:51:03.346Z [INFO] (Copier) smarthub-greengrass-nodejs: Startup script exited. {exitCode=1, serviceInstance=1, serviceName=smarthub-greengrass-nodejs, currentState=STARTING}

2022-06-23T02:51:03.352Z [INFO] (pool-2-thread-16) smarthub-greengrass-nodejs: shell-runner-start. {scriptName=services.smarthub-greengrass-nodejs.lifecycle.shutdown.script, serviceInstance=1, serviceName=smarthub-greengrass-nodejs, currentState=BROKEN, command=["/greengrass/v2/packages/artifacts/aws.greengrass.LambdaLauncher/2.0.10/lambda-..."]}

2022-06-23T02:51:03.387Z [WARN] (Copier) smarthub-greengrass-nodejs: stderr. 2022/06/23 02:51:03 could not read process state file /greengrass/v2/work/smarthub-greengrass-nodejs/work/worker/P7QmJu--20Asa2ep8zLcZYL4TPTGKRFtbfWy1UYUcsM/state.json: open /greengrass/v2/work/smarthub-greengrass-nodejs/work/worker/P7QmJu--20Asa2ep8zLcZYL4TPTGKRFtbfWy1UYUcsM/state.json: no such file or directory. {scriptName=services.smarthub-greengrass-nodejs.lifecycle.shutdown.script, serviceInstance=1, serviceName=smarthub-greengrass-nodejs, currentState=BROKEN}
`

To Reproduce
Follow these steps to launch greengrass via docker (https://docs.aws.amazon.com/greengrass/v2/developerguide/run-greengrass-docker-automatic-provisioning.html)

Once you have the docker container running from the docker-compose file provision a nodejs12.x lambda component.

{ "lambdaExecutionParameters": { "EnvironmentVariables": {} }, "containerParams": { "memorySize": 16000, "mountROSysfs": false, "volumes": {}, "devices": {} }, "containerMode": "NoContainer", "timeoutInSeconds": 3, "maxInstancesCount": 100, "inputPayloadEncodingType": "json", "maxQueueSize": 1000, "pinned": false, "maxIdleTimeInSeconds": 60, "statusTimeoutInSeconds": 60, "pubsubTopics": { "0": { "topic": "cloud", "type": "IOT_CORE" } } }

Send a message to the topic

Get error.

Expected behavior
lambda should run

Actual behavior
lambda fails to run with error posted above.

Environment
GREENGRASS_RELEASE_VERSION=2.5.3