aws-ia / cfn-ecr-aws-soci-index-builder Goto Github PK

Hello Team,

Please enable this task for the UAE region, because i'm trying to provision this to the UAE region me-central-1 unfortunately cloudformation is failing. while checking cfn-ecr-aws-soci-index-builder/.taskcat.yml I found UAE region is not enabled yet. Please enable this i have provisioned my docker image which has nearly 2 GB So SOCi will help us speed up the container provision in AWS ECS Fargate. Look forward to hearing from you soon.

Kind Regards
Thank you

Problem

v0.4.1 of SOCI has been released. We should upgrade to it.

Expected Behavior

The CFN solution should try to use the latest available release.

Current Behavior

We are using v0.4.0.

Possible Solution

Upgrade to patch release.

Steps to Reproduce (if applicable)

If you're reporting a bug, please provide steps to reproduce the issue.

Screenshots (if applicable)

If you have any relevant screenshots, please attach them here.

Environment

Please provide information about the environment in which you're experiencing the issue, including:

• Project version:
• Operating system:
• Browser version (if applicable):
• Any relevant hardware or software information:

Additional Context

Please provide any additional information or context that may be helpful in resolving the issue or implementing the requested feature.

Hello,

I think it'd be a good idea to have this published in AWS Serverless Application Repository to make users deployment of this almost easier. Plus, it has the benefit to give you out-of-the box analytics counting the number of deployments ;)

Thanks for your consideration,

The soci cli allows you to pass in a --min-layer-size, it would be great if you can set this as a parameter in the cloudformation template (admittedly it would set this configuration for all indexes which may or may not be desired).

Hello Team,

We are facing issues in use region me-central-1, still we cannot provision your CloudFormation stack, kindly fix this permission issues from your end. The stack I can successfully provision other region without any permission issues.
The UAE region 3 week before I had given pull request to the .taskcat.yml file your team approved and I can able to see in the master branch.
I guess your s3 bucket CloudFormation template allow permission for use region as well.
Looking forward your kind update as soon as possible.

Bellow ERROR getting while provision stack on uae region:-
Resource handler returned message: "Your access has been denied by S3, please make sure your request credentials have permission to GetObject for aws-quickstart-me-central-1/cfn-ecr-aws-soci-index-builder/functions/packages/soci-index-generator-lambda/soci_index_generator_lambda.zip. S3 Error Code: AccessDenied. S3 Error Message: Access Denied (Service: Lambda, Status Code: 403, Request ID:

Problem

This runtime generates SOCI indexes that suffer from a bug whereby a copy of a file will not complete. The files generated are sparse.

Current Behavior

The image when loaded has many sparse files that a simple cp command will stall on.

Steps to Reproduce (if applicable)

With the following simple Docker file

`FROM oraclelinux:9

RUN dnf install -y libreoffice
&& echo "Hello world" > /tmp/test.txt
&& groupadd -g 679 convert
&& useradd -d /opt/noggin.io/soffice -u 679 convert -g 679

USER convert

RUN soffice --convert-to pdf --outdir /tmp /tmp/test.txt

CMD cp /opt/noggin.io/soffice/.config/libreoffice/4/user/basic/Standard/Module1.xba /tmp/Module1.xba.copied
`

Build an image, use this generator on it and create an ECS task.

This task will continue to run as the cp command does not complete.

Without the SOCI index, the task will quit immediately.

If the soci binary is used instead to build the SOCI index, the bug will not occur

Additional Context

Running the following in an ECS task for an image with the SOCI index

find / -type f -printf "%S\t%p\n" | gawk '$1 < 1.0 {print}

Any file that reports as less than 50% non sparse the cp command will stall on

Problem

The problem lies that the snapshotter template is available on s3 on a single URL. Our team has experienced vast improvements to performance of SOCI images when updating the template from the July version to November version. However to have the template deployed to production we are using terraform. Since the state of the Cloudformation stack does not change, the stack is never updated (technically we can destroy it and redeploy).

Expected Behavior

We would love to see some versioned endpoints of the template, so that we can put that URL in our IaC and when an update comes, the version can be changed in template_url.

Problem

The SOCI CLI sets a 10 MiB default --min-layer-size threshold when building indexes, which means layers under 10 MiB will not get zTOCs created for them. The index builder sets the min-layer-size to 0 -

Expected Behavior

The SOCI CLI and Index Builder should have the same defaults for creating indexes.

Current Behavior

Different min-layer-size for both the SOCI CLI and index builder.

Possible Solution

The min-layer-size default should be the same for the CLI and the index builder.

Steps to Reproduce (if applicable)

If you're reporting a bug, please provide steps to reproduce the issue.

Screenshots (if applicable)

If you have any relevant screenshots, please attach them here.

Environment

Please provide information about the environment in which you're experiencing the issue, including:

• Project version:
• Operating system:
• Browser version (if applicable):
• Any relevant hardware or software information:

Additional Context

Please provide any additional information or context that may be helpful in resolving the issue or implementing the requested feature.

Just a thought regarding the existing .taskcat configuration file in the repo: it includes regions, i.e us-gov ones which most people who might contribute do not have access to.
Might be worth having 2 project files, and adapt your automated testing to cover all regions, and have contributors cover the "top 10 regions".

Problem

AWS warned that the Go 1.x runtime will be discontinued. I then tried to update the CloudFormation and was happy to see it was modified to change the Go lambda runtime to al2. However, it does not seem to be working.

Expected Behavior

The cfn-ecr-aws-soci-index-bu-SociIndexGeneratorLambda should run without errors.

Current Behavior

From CloudWatch, this is what I get:

RequestId: f1ccceed-30f9-4f02-a4e8-6f2a40714e75 Error: Couldn't find valid bootstrap(s): [/var/task/bootstrap /opt/bootstrap] Runtime.InvalidEntrypoint

Hello,

since AWS is deprecating go1.x Lambda runtime at the end of this year I am wondering if we can expect to switch runtime environment for SociIndexGenerator function on the repo to use provided.al2.

Kind regards

Problem

#45 switches the builder to use the default min-layer-size (10MiB from 0. This behavior will have an impact on customers who've used the solution with smaller images (or more specifically images where each layer is under 10MiB). We should update the documentation to include a blurb about the possibility of no index being pushed depending on image size and the general recommendation that larger images (generally >250MiB) see the most benefit with SOCI.

Expected Behavior

Current Behavior

Please describe what currently happens, or how the lack of this feature affects your experience with the project.

Possible Solution

Documentation explaining the possibility of no index being pushed.

Steps to Reproduce (if applicable)

If you're reporting a bug, please provide steps to reproduce the issue.

Screenshots (if applicable)

If you have any relevant screenshots, please attach them here.

Environment

Please provide information about the environment in which you're experiencing the issue, including:

• Project version:
• Operating system:
• Browser version (if applicable):
• Any relevant hardware or software information:

Additional Context

Please provide any additional information or context that may be helpful in resolving the issue or implementing the requested feature.