Code Monkey home page Code Monkey logo

cdk-serverless-clamscan's Introduction

cdk-serverless-clamscan

npm version PyPI version

An aws-cdk construct that uses ClamAV® to scan newly uploaded objects to Amazon S3 for viruses. The construct provides a flexible interface for a system to act based on the results of a ClamAV virus scan. Check out this blogpost for a guided walkthrough.

Overview

Pre-Requisites

Docker: The ClamAV Lambda functions utilizes a container image that is built locally using docker bundling

Examples

This project uses projen and thus all the constructs follow language specific standards and naming patterns. For more information on how to translate the following examples into your desired language read the CDK guide on Translating TypeScript AWS CDK code to other languages

Example 1. (Default destinations with rule target)

typescript

import { RuleTargetInput } from 'aws-cdk-lib/aws-events';
import { SnsTopic } from 'aws-cdk-lib/aws-events-targets';
import { Bucket } from 'aws-cdk-lib/aws-s3';
import { Topic } from 'aws-cdk-lib/aws-sns';
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { ServerlessClamscan } from 'cdk-serverless-clamscan';

export class CdkTestStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const bucket_1 = new Bucket(this, 'rBucket1');
    const bucket_2 = new Bucket(this, 'rBucket2');
    const bucketList = [bucket_1, bucket_2];
    const sc = new ServerlessClamscan(this, 'rClamscan', {
      buckets: bucketList,
    });
    const bucket_3 = new Bucket(this, 'rBucket3');
    sc.addSourceBucket(bucket_3);
    const infectedTopic = new Topic(this, 'rInfectedTopic');
    sc.infectedRule?.addTarget(
      new SnsTopic(infectedTopic, {
        message: RuleTargetInput.fromEventPath(
          '$.detail.responsePayload.message',
        ),
      }),
    );
  }
}

python

from aws_cdk import (
  Stack,
  aws_events as events,
  aws_events_targets as events_targets,
  aws_s3 as s3,
  aws_sns as sns
)
from cdk_serverless_clamscan import ServerlessClamscan
from constructs import Construct

class CdkTestStack(Stack):

  def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
    super().__init__(scope, construct_id, **kwargs)

    bucket_1 = s3.Bucket(self, "rBucket1")
    bucket_2 = s3.Bucket(self, "rBucket2")
    bucketList = [ bucket_1, bucket_2 ]
    sc = ServerlessClamscan(self, "rClamScan",
      buckets=bucketList,
    )
    bucket_3 = s3.Bucket(self, "rBucket3")
    sc.add_source_bucket(bucket_3)
    infected_topic = sns.Topic(self, "rInfectedTopic")
    if sc.infected_rule != None:
      sc.infected_rule.add_target(
        events_targets.SnsTopic(
          infected_topic,
          message=events.RuleTargetInput.from_event_path('$.detail.responsePayload.message'),
        )
      )

Example 2. (Bring your own destinations)

typescript

import {
  SqsDestination,
  EventBridgeDestination,
} from 'aws-cdk-lib/aws-lambda-destinations';
import { Bucket } from 'aws-cdk-lib/aws-s3';
import { Queue } from 'aws-cdk-lib/aws-sqs';
import { Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { ServerlessClamscan } from 'cdk-serverless-clamscan';

export class CdkTestStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const bucket_1 = new Bucket(this, 'rBucket1');
    const bucket_2 = new Bucket(this, 'rBucket2');
    const bucketList = [bucket_1, bucket_2];
    const queue = new Queue(this, 'rQueue');
    const sc = new ServerlessClamscan(this, 'default', {
      buckets: bucketList,
      onResult: new EventBridgeDestination(),
      onError: new SqsDestination(queue),
    });
    const bucket_3 = new Bucket(this, 'rBucket3');
    sc.addSourceBucket(bucket_3);
  }
}

python

from aws_cdk import (
  Stack,
  aws_lambda_destinations as lambda_destinations,
  aws_s3 as s3,
  aws_sqs as sqs
)
from cdk_serverless_clamscan import ServerlessClamscan
from constructs import Construct

class CdkTestStack(Stack):

  def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
    super().__init__(scope, construct_id, **kwargs)

    bucket_1 = s3.Bucket(self, "rBucket1")
    bucket_2 = s3.Bucket(self, "rBucket2")
    bucketList = [ bucket_1, bucket_2 ]
    queue = sqs.Queue(self, "rQueue")
    sc = ServerlessClamscan(self, "rClamScan",
      buckets=bucketList,
      on_result=lambda_destinations.EventBridgeDestination(),
      on_error=lambda_destinations.SqsDestination(queue),
    )
    bucket_3 = s3.Bucket(self, "rBucket3")
    sc.add_source_bucket(bucket_3)

Operation and Maintenance

When ClamAV publishes updates to the scanner you will see “Your ClamAV installation is OUTDATED” in your scan results. While the construct creates a system to keep the database definitions up to date, you must update the scanner to detect all the latest Viruses.

Update the docker images of the Lambda functions with the latest version of ClamAV by re-running cdk deploy.

API Reference

See API.md.

Contributing

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

cdk-serverless-clamscan's People

Contributors

amazon-auto avatar awskaran avatar bibliotechy avatar dependabot[bot] avatar dontirun avatar felix-iw avatar github-actions[bot] avatar houstonj1 avatar jpduckwo avatar lirwin3007 avatar michelesr avatar skim-vivlio avatar vmarinescu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

cdk-serverless-clamscan's Issues

No scan tag should be considered in the policy

When the file is just added the tag doesn't exist yet. And the file can be downloaded. Which can be a potential problem if the file is infected.

Suggestion: handle the empty tag as InProgress and disallow download in the policy

Recommendation for scanning S3 bucket on a schedule

Hi 👋! I'm a little new to the world of CDK so this may be a dumb question. I'm wanting to rescan all objects in a bucket if virus definitions update and monthly. What would be the recommended approach in accomplishing this? I'm currently using the setup mentioned in this blog: https://aws.amazon.com/blogs/developer/virus-scan-s3-buckets-with-a-serverless-clamav-based-cdk-construct/

Any nudges in the right direction would be helpful and appreciated 🙂

Using named profiles with "env" does not seem to work

As soon as I add the construct to my project, I get this error message: Error at /MainServiceStack/ClamAVStack] Need to perform AWS calls for account 123, but the current credentials are for 456 where 123 is the account I have configured in my profile and set up using "env" in my main cdk stack and 456 is the account that my user lives in. For all actions I have to assume a role in another account (e.g. production, staging, development) to be able to deploy to that account. It all works fine but not in combination with this construct. Has anyone seen this before or have any ideas how to fix it?

Thanks

Files with prefixes are not processed properly

As mentioned in #54 (comment) files with prefixes fail to scan

[ERROR] FileNotFoundError: [Errno 2] No such file or directory: '/mnt/lambda/b0289808-bb0c-4d94-934f-d08e8c27ad2a/nestedfoldercheck/0000001.jpg.262a6F0b'
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 347, in decorate
    return lambda_handler(event, context)
  File "/var/task/lambda.py", line 75, in lambda_handler
    download_object(input_bucket, input_key, payload_path)
  File "/var/task/lambda.py", line 119, in download_object
    s3_resource.Bucket(input_bucket).download_file(
  File "/var/lang/lib/python3.8/site-packages/boto3/s3/inject.py", line 244, in bucket_download_file
    return self.meta.client.download_file(
  File "/var/lang/lib/python3.8/site-packages/boto3/s3/inject.py", line 170, in download_file
    return transfer.download_file(
  File "/var/lang/lib/python3.8/site-packages/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/var/lang/lib/python3.8/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/var/lang/lib/python3.8/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/var/lang/lib/python3.8/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/download.py", line 571, in _main
    fileobj.seek(offset)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/utils.py", line 367, in seek
    self._open_if_needed()
  File "/var/lang/lib/python3.8/site-packages/s3transfer/utils.py", line 350, in _open_if_needed
    self._fileobj = self._open_function(self._filename, self._mode)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/utils.py", line 261, in open
    return open(filename, mode)

Scan and Definition download Lambda's fail when built on Arm64

Hi,
I'm getting an error when downloading virus definitions and was hoping that someone could shine a light on what could be wrong.
I have created a stack from the examples, but using my own existing bucket as the source. No other changes.
I'm using version 1.1.51 with @aws-cdk 1.125.0.

When I deploy, I get this error:
Received response status [FAILED] from custom resource. Message returned: Initial definition download failed: Unhandled

The stack is deleted, but the logs are still there.

The above error comes from the log group: /aws/lambda/ClamScanStack-ClamScanInitDefsxxxxxxxxx
And here's the error:

[ERROR]	2022-04-15T07:57:58.540Z	3eaf3937-3fb7-42df-8500-850e3e4d702f	Initial definition download failed: Unhandled
--
[INFO]	2022-04-15T07:57:58.541Z	3eaf3937-3fb7-42df-8500-850e3e4d702f	https://cloudformation-custom-resource-response-eunorth1.s3.eu-north-1.amazonaws.com/<REMOVED>
[INFO]	2022-04-15T07:57:58.541Z	3eaf3937-3fb7-42df-8500-850e3e4d702f	Response body:
[INFO]	2022-04-15T07:57:58.544Z	3eaf3937-3fb7-42df-8500-850e3e4d702f	{     "Status": "FAILED",     "Reason": "Initial definition download failed: Unhandled",     "PhysicalResourceId": "2022/04/15/[$LATEST]e408d585eda146779e24ff186c527314",     "StackId": "arn:aws:cloudformation:eu-north-1:369139281633:stack/ClamScanStack/<REMOVED>",     "RequestId": "06ef750a-bb2f-492e-bbc5-06ad1d4757d2",     "LogicalResourceId": "ClamScanInitDefsCrxxxxxxx",     "NoEcho": false,     "Data": {} }
[INFO]	2022-04-15T07:57:58.756Z	3eaf3937-3fb7-42df-8500-850e3e4d702f	Status code: 200
END RequestId: 3eaf3937-3fb7-42df-8500-850e3e4d702f

The real error though seems to happen during download. The logs from /aws/lambda/ClamScanStack-ClamScanDownloadDefsxxxxxxxx can be seen here:

IMAGE	Launch error: fork/exec /var/lang/bin/python3: exec format error	Entrypoint: [/var/lang/bin/python3,-m,awslambdaric]	Cmd: [lambda.lambda_handler]	WorkingDir: [/var/task]
--
START RequestId: 26693538-738d-424d-9c31-ca8c1b221a6a Version: $LATEST
IMAGE	Launch error: fork/exec /var/lang/bin/python3: exec format error	Entrypoint: [/var/lang/bin/python3,-m,awslambdaric]	Cmd: [lambda.lambda_handler]	WorkingDir: [/var/task]
END RequestId: 26693538-738d-424d-9c31-ca8c1b221a6a
REPORT RequestId: 26693538-738d-424d-9c31-ca8c1b221a6a	Duration: 2.88 ms	Billed Duration: 3 ms	Memory Size: 1024 MB	Max Memory Used: 3 MB
RequestId: 26693538-738d-424d-9c31-ca8c1b221a6a Error: fork/exec /var/lang/bin/python3: exec format errorRuntime.InvalidEntrypoint

Any ideas if this could be a problem with the package/docker image or a problem on my side?
I'm on a Macbook Pro with M1.

Source buckets with nested folder directories and large zip files fail to scan

When testing a deployment of this construct, I tested the source bucket with 3 different configurations. 1 with a relatively large zip file (7.5 GB), 1 with a smaller zip file (1.5 GB), and then a nested structure of images (each image about 17kb).

The smaller (1.5 GB) zip files and individual images, when placed at the root of the bucket, passed the scan.

The larger zip files and the nested structures returned the following errors respectively.....

[ERROR] FileNotFoundError: [Errno 2] No such file or directory: '7za'
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 347, in decorate
    return lambda_handler(event, context)
  File "/var/task/lambda.py", line 76, in lambda_handler
    expand_if_large_archive(
  File "/var/task/lambda.py", line 142, in expand_if_large_archive
    archive_summary = subprocess.run(
  File "/var/lang/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/var/lang/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/var/lang/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
[ERROR] FileNotFoundError: [Errno 2] No such file or directory: '/mnt/lambda/b0289808-bb0c-4d94-934f-d08e8c27ad2a/nestedfoldercheck/0000001.jpg.262a6F0b'
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 347, in decorate
    return lambda_handler(event, context)
  File "/var/task/lambda.py", line 75, in lambda_handler
    download_object(input_bucket, input_key, payload_path)
  File "/var/task/lambda.py", line 119, in download_object
    s3_resource.Bucket(input_bucket).download_file(
  File "/var/lang/lib/python3.8/site-packages/boto3/s3/inject.py", line 244, in bucket_download_file
    return self.meta.client.download_file(
  File "/var/lang/lib/python3.8/site-packages/boto3/s3/inject.py", line 170, in download_file
    return transfer.download_file(
  File "/var/lang/lib/python3.8/site-packages/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/var/lang/lib/python3.8/site-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/var/lang/lib/python3.8/site-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/var/lang/lib/python3.8/site-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/download.py", line 571, in _main
    fileobj.seek(offset)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/utils.py", line 367, in seek
    self._open_if_needed()
  File "/var/lang/lib/python3.8/site-packages/s3transfer/utils.py", line 350, in _open_if_needed
    self._fileobj = self._open_function(self._filename, self._mode)
  File "/var/lang/lib/python3.8/site-packages/s3transfer/utils.py", line 261, in open
    return open(filename, mode)

Help with using imported buckets

Hi,

Been following the blog for the s3Scanner and completed all the steps but when I upload an object, I am getting a Status "Error" on the tag. Here is the cloudwatch metrics:
[ERROR] Exception: {
"source": "serverless-clamscan",
"input_bucket": "bucket",
"input_key": "sample-virus.txt",
"status": "ERROR",
"message": "Forbidden"
}
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 354, in decorate
    return lambda_handler(event, context, *args, **kwargs)
  File "/var/task/lambda.py", line 78, in lambda_handler
    download_object(input_bucket, input_key, payload_path)
  File "/var/task/lambda.py", line 149, in download_object
    report_failure(
  File "/var/task/lambda.py", line 299, in report_failure
    raise Exception(json.dumps(exception_json))

Sample virus was from the blog too. Also asked assistance from AWS Support but we still could not fix the issue. I hope you can help us with this. Thank you.

Does not work with existing buckets

the solution doesn't seem to work with existing buckets. Our workaround is to change the signature of the addSourceBucket method and cast:

  addSourceBucket(bucket: IBucket) {
    this._scanFunction.addEventSource(
      new S3EventSource(bucket as Bucket, { events: [EventType.OBJECT_CREATED] }),
  // ...

Unable to remove imported bucket

Summary

After specifying an existing bucket as a scanning target, I attempted to remove it and was met with the following error: An error occurred (AccessDenied) when calling the GetBucketNotificationConfiguration operation: Access Denied.

Disclaimer -- this is running in an enterprise type environment in gov cloud and I'm not savvy enough to understand the permissions boundaries and the policies granted to the roles I have access to - just reporting this based on what I've been told. That said, here is what I've attempted. Note that my pseudo-code below does not include the Aspects that are unique to this environment.

Import / add an existing bucket to be scanned

from aws_cdk import (
  Stack,
  aws_events as events,
  aws_events_targets as events_targets,
  aws_s3 as s3,
  aws_sns as sns,
  DefaultStackSynthesizer,
  App,
)
from cdk_serverless_clamscan import ServerlessClamscan
from constructs import Construct


class CdkTestStack(Stack):

  def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
    super().__init__(scope, construct_id, **kwargs)

    bucket_1 = s3.Bucket(self, "rBucket1")
    bucket_2 = s3.Bucket(self, "rBucket2")
    bucketList = [ bucket_1, bucket_2 ]
    sc = ServerlessClamscan(self, "rClamScan",
      buckets=bucketList,
      accept_responsibility_for_using_imported_bucket=True,
    )
    bucket = s3.Bucket.from_bucket_name(self, 'MyImportedBucket', 'an-existing-bucket');
    sc.add_source_bucket(bucket);

    bucket_policy_for_specific_imported_bucket = s3.BucketPolicy(self, 'BucketPolicy', bucket=bucket)
    bucket_policy_for_specific_imported_bucket.document.add_statements(sc.get_policy_statement_for_bucket(bucket))

    infected_topic = sns.Topic(self, "rInfectedTopic")
    if sc.infected_rule != None:
      sc.infected_rule.add_target(
        events_targets.SnsTopic(
          infected_topic,
          message=events.RuleTargetInput.from_event_path('$.detail.responsePayload.message'),
        )
      )

app = App()
stack = CdkTestStack(app, "clamav-scan-stack")
app.synth()

This successfully setup an-existing-bucket to be scanned after running cdk deploy.

Remove bucket as a scan target

Now, I'd like to remove an-existing-bucket from the scan targets. My expectation was that I could just update app.py, remove the references that added it, re-run cdk deploy and the bucket policy would be removed, etc.

from aws_cdk import (
  Stack,
  aws_events as events,
  aws_events_targets as events_targets,
  aws_s3 as s3,
  aws_sns as sns,
  DefaultStackSynthesizer,
  App,
)
from cdk_serverless_clamscan import ServerlessClamscan
from constructs import Construct


class CdkTestStack(Stack):

  def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
    super().__init__(scope, construct_id, **kwargs)

    bucket_1 = s3.Bucket(self, "rBucket1")
    bucket_2 = s3.Bucket(self, "rBucket2")
    bucketList = [ bucket_1, bucket_2 ]
    sc = ServerlessClamscan(self, "rClamScan",
      buckets=bucketList,
      accept_responsibility_for_using_imported_bucket=True,
    )

    infected_topic = sns.Topic(self, "rInfectedTopic")
    if sc.infected_rule != None:
      sc.infected_rule.add_target(
        events_targets.SnsTopic(
          infected_topic,
          message=events.RuleTargetInput.from_event_path('$.detail.responsePayload.message'),
        )
      )

app = App()
stack = CdkTestStack(app, "clamav-scan-stack")
app.synth()

Now cdk deploy gives me an error about lacking permissions for GetBucketNotificationConfiguration.

Someone smarter than me has provided this analysis:

It seems like the permissions for the Role attached to the "clamav-scan-stack-BucketNotificationsHandler050a05-QNovedi171qG" lambda (service-clamav-scan-stack-BucketNotificationsHandler050a0587b75) doesn't have permissions for s3:GetBucketNotificationConfiguration. It only has s3:PutBucketNotification, so it's not limited by the permissions boundary.

After the s3:GetBucketNotificationConfiguration permission was added to the role, I was able to remove the imported bucket successfully.

Thanks for taking a look at this!

Dotnet support

Hi, I wanted to be able to integrate this with my CDK stack however I can't seem to get projen to generate a C# package. I've added to the .projenrc.js as seen on my fork https://github.com/awslabs/cdk-serverless-clamscan/compare/main...robert-lilleker:dotnet-support?expand=1 however I get the following message:

[jsii-pacmak] [WARN] Exception occurred, not cleaning up [jsii-pacmak] [WARN] dotnet failed TypeError: Cannot read properties of undefined (reading 'namespace') at DotNetTypeResolver.resolveNamespacesDependencies (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/targets/dotnet/dotnettyperesolver.js:71:46) at DotNetGenerator.generate (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/targets/dotnet/dotnetgenerator.js:43:27) at Dotnet.generateCode (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/target.js:29:28) at async DotnetBuilder.generateModuleCode (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/targets/dotnet.js:94:9) at async /home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/targets/dotnet.js:62:30 at async Function.make (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/util.js:227:36) at async DotnetBuilder.buildModules (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/targets/dotnet.js:35:35) at async Promise.all (index 0) at async Object.pacmak (/home/ec2-user/environment/forks/cdk-serverless-clamscan/node_modules/jsii-pacmak/lib/index.js:61:9) 👾 Task "release » build » package" failed when executing "jsii-pacmak" (cwd: /home/ec2-user/environment/forks/cdk-serverless-clamscan) error Command failed with exit code 1.

Disable Access Logs?

Hi is there a way to disable access logs from writing to a bucket? I noticed that there are a lot of logs generated over time and I was wondering if there was a way to make it optional.

Also, what kind of data is being stored in here?

Thank you in advance!

Error `FreshClam exited with unexpected code: 14` in freshclam_update

Hello, I'm using this great construct in a stack, and the scanning functionality is working properly, but the definition update seems to be failing with an unexpected exit code. Looking at the man page for freshclam, this exit code is not documented.

[ERROR] Exception: 
{
    "source": "serverless-clamscan-update",
    "message": "FreshClam exited with unexpected code: 14"
}

Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 347, in decorate
    return lambda_handler(event, context)
  File "/var/task/lambda.py", line 38, in lambda_handler
    freshclam_update(download_path)
  File "/var/task/lambda.py", line 101, in freshclam_update
    report_failure(e.message)
  File "/var/task/lambda.py", line 111, in report_failure
    raise Exception(json.dumps(exception_json))
[ERROR] Exception: {"source": "serverless-clamscan-update", "message": "FreshClam exited with unexpected code: 14"} Traceback (most recent call last):   File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 347, in decorate     return lambda_handler(event, context)   File "/var/task/lambda.py", line 38, in lambda_handler     freshclam_update(download_path)   File "/var/task/lambda.py", line 101, in freshclam_update     report_failure(e.message)   File "/var/task/lambda.py", line 111, in report_failure     raise Exception(json.dumps(exception_json))

How to override AWS:::IAM:::Role names & opts?

Hi there! New user of AWS CDK so apologize if this is something I've overlooked... I'm in an "unusual" environment where role names absolutely must be prefixed with "service-" and additionally I must attached a PermissionsBoundary during role creation.

To start, I've done the following:

cdk bootstrap --show-template > template.yaml
vi template.yaml # prefix Role names with service, set PermissionsBoundary to arn of the boundary
cdk bootstrap --template template.yaml

Then I've also had to provide these new names to the stack constructor, via the synthesizer=DefaultSynthesizer(deploy_role_arn=... arguments.

Now when I am attempting cdk deploy I make good progress until it looks like NEW roles are being created. Here's an example, with account IDs and other bits redacted:

6:40:56 PM | CREATE_FAILED        | AWS::IAM::Role                        | rClamScanScanVPCFlowLogsIAM
Role3049EDC1
API: iam:CreateRole User: arn:aws-us-gov:sts::003223421170:assumed-role/service-cdk-hnb659fds-cfn-exec-
role-003223421170/AWSCloudFormation is not authorized to perform: iam:CreateRole on resource: arn:aws-u
s-gov:iam::003223421170:role/clamav-scan-stack-rClamScanScanVPCFlowLogsIAMRole3-KIK2YAJHSF3D because no
permissions boundary allows the iam:CreateRole action

So, any suggestions how I can get the 'role/clamav-scan' (and others) to contain my "service-" prefix and also attach the PermissionsBoundary?

ERROR status when using permissions boundary with scan lambda IAM role

Might be related to #531 but not quite sure.

If IAM role used by scanning lambda function has any permissions boundary attached, even if it's allow all, the scan fails with an error:

[ERROR] Exception: 
{
    "source": "serverless-clamscan",
    "input_bucket": "<REMOVED>",
    "input_key": "<REMOVED>",
    "status": "ERROR",
    "message": "Forbidden"
}

Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 354, in decorate
    return lambda_handler(event, context, *args, **kwargs)
  File "/var/task/lambda.py", line 78, in lambda_handler
    download_object(input_bucket, input_key, payload_path)
  File "/var/task/lambda.py", line 149, in download_object
    report_failure(
  File "/var/task/lambda.py", line 299, in report_failure
    raise Exception(json.dumps(exception_json))

Steps to reproduce

  1. Add the following permissions boundary to the IAM role
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": "*",
            "Resource": "*",
            "Effect": "Allow"
        }
    ]
}
  1. Upload the object to the bucket

To add to this, if I remove the deny bucket policy added by ClamAV, this fixes the error as well, even if permissions boundary is still in place. I suspect that has something to do with the identity of the IAM roles that have permission boundaries attached.

Expected behaviour

I expect ClamAV scan to support IAM permission boundaries. Permission boundaries enforced on the stack level is the common security best practice.

Scan fails with error code -9

Hi,
I've been running clamscan in my ubuntu machine for the past few weeks and I've been able to run it successfully. However, when I try to run the same using the docker container using the dockerfile in AWS lambda, I keep running into error code -9. There is no specific reason as to why this is failing. I'm attaching the exception stack below:

[ERROR] Exception: 
{
    "source": "serverless-clamscan",
    "input_bucket": "chatbot-nlu-models",
    "input_key": "clamscan/test1.log",
    "status": "ERROR",
    "message": "ClamAV exited with unexpected code: -9.\nOutput: "
}

Traceback (most recent call last):
  File "/var/task/lambda.py", line 80, in lambda_handler
    summary = scan(
  File "/var/task/lambda.py", line 289, in scan
    report_failure(input_bucket, input_key, download_path, e.message)
  File "/var/task/lambda.py", line 203, in report_failure
    raise Exception(json.dumps(exception_json))

BUG : Virus definition logging bucket does not allow passing an imported bucket

What is the problem?

Cdk Serverless Clamscan construct does not accept an imported Bucket resource passed as a target for access logs created for Virus Definition bucket.

jsii.errors.JSIIError: Value did not match any type in union: Expected a boolean, got {"$jsii.byref":"aws-cdk-lib.aws_s3.BucketBase@10006"}, Object of type aws-cdk-lib.aws_s3.BucketBase is not convertible to aws-cdk-lib.aws_s3.Bucket

Reproduction Steps

#!/usr/bin/env python3
import aws_cdk as cdk
from aws_cdk import Aspects
from aws_cdk import aws_s3 as s3
from cdk_nag import AwsSolutionsChecks
from cdk_serverless_clamscan import ServerlessClamscan, ServerlessClamscanLoggingProps
from constructs import Construct

class BucketStack(cdk.Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        s3_access_logs_bucket = s3.Bucket(
            self,
            "rS3AccessLogsBucket",
            block_public_access=s3.BlockPublicAccess.BLOCK_ALL,
            bucket_name=f"s3-access-logs-{self.account}-{self.region}",
            encryption=s3.BucketEncryption.S3_MANAGED,
            enforce_ssl=True,
            versioned=True,
        )

        cdk.CfnOutput(
            self,
            "oS3AccessLogsBucketArn",
            export_name="oS3AccessLogsBucketArn",
            value=s3_access_logs_bucket.bucket_arn,
        )


class AntiVirusStack(cdk.Stack):
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        s3_access_logs_bucket = s3.Bucket.from_bucket_arn(
            self,
            "rS3AccessLogsBucket",
            bucket_arn=cdk.Fn.import_value("oS3AccessLogsBucketArn"),
        )

        ServerlessClamscan(
            self,
            "antivirus-scanning",
            defs_bucket_access_logs_config=ServerlessClamscanLoggingProps(
                logs_bucket=s3_access_logs_bucket,
                logs_prefix=f"s3-access-logs-{self.account}-{self.region}",
            ),
        )


app = cdk.App()
BucketStack(app, "BucketStack")
AntiVirusStack(app, "AntiVirusStack")

Aspects.of(app).add(AwsSolutionsChecks())
app.synth()

What did you expect to happen?

I expected the cdk construct to accept an imported bucket being passed as a target for access logging for the Virus Definitions bucket

cdk-serverless-clamscan version

2.1.26

Language

Python

Other information

Optionally allow access to non-clean files

I am evaluating this construct for use in an application that needs to store user-submitted files.
It is possible that a business requirement to allow skipping "INFECTED" status on a file can arise. Is it ok add a possibility to skip adding "DENY" resource when adding a source bucket?

I can implement that contribution myself if possible.

An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type..

I'm running into this error when running cdk. Sorry for being silly, but how I can update or configure new bucket notification prefix if having conflict??

CdkClamAVStack | 55/58 | 9:02:31 AM | CREATE_FAILED | Custom::S3BucketNotifications Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type..

Current Code:
`
export class CdkClamAVStack extends Stack {

constructor(scope: Construct, id: string, props?: StackProps) {

super(scope, id, props);

const directBucket = Bucket.fromBucketName(
  this,
  process.env.DATA_BUCKET || `havendirect-${process.env.ENV_TIER}-data`,
  process.env.DATA_BUCKET || `havendirect-${process.env.ENV_TIER}-data`,
);

const sc = new ServerlessClamscan(this, 'rClamscan', {
  acceptResponsibilityForUsingImportedBucket: true,
  buckets: [directBucket],
});

new CfnOutput(this, 'oBucketName', {
  description: 'The name of the input S3 Bucket',
  value: directBucket.bucketName
})

}
}
`

Need GovCloud Workaround For LogGroup Tagging Support

Following:
Steps reproducible from the original blog.

Seems related to CDK Issue # 17960

CDK Version in use: 2.38.1
Version of cdk-serverless-clamscan construct: 2.4.4 latest at the time of posting via package.json's self minor updates.

Error Output deploying with cdk to us-gov-west-1:

11:08:21 AM | CREATE_FAILED | AWS::Logs::LogGroup | rClamscanScanVPCFlowLogsLogGroupXXXXXXX
Encountered unsupported property Tags

❌ CdkTestStack failed: Error: The stack named CdkTestStack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Encountered unsupported property Tags
at prepareAndExecuteChangeSet (/opt/homebrew/lib/node_modules/aws-cdk/lib/api/deploy-stack.ts:386:13)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at CdkToolkit.deploy (/opt/homebrew/lib/node_modules/aws-cdk/lib/cdk-toolkit.ts:219:24)
at initCommandLine (/opt/homebrew/lib/node_modules/aws-cdk/lib/cli.ts:347:12)

The stack named CdkTestStack failed creation, it may need to be manually deleted from the AWS console: ROLLBACK_COMPLETE: Encountered unsupported property Tags

Allow delete bucket on cloudFormation stack rollback

Hello,

I'm trying this scanner solution and it works well except that if my stack encounter an issue (I raised VPC instance limit) so cloudFormation tried to rollback my stack. However this rollback blocked and I can see error message trying to delete rClamScanVirusDefsBucket and sometimes policy as well.
In cloudFormation context, all ressources should be deletable.

Side note: it looks like scanner take 20 to 30 sec per file. Can you confirm me if it's normal performance? I set scanFunctionMemorySize to 10GB and reservedConcurrency to 3.

Thx

403 error on freshclaim command

I have the Private Mirror pointing to the env variable DEFS_URL (the virus def. s3 bucket).
When freshclam_update runs, it fails with an 403 error, when trying to download the definitions from the s3 bucket.
Any idea?

Allow using existing VPC

By default ServerlessClamscan creates a new VPC with isolated subnets, but in some scenarios users might want to reuse existing VPC (for example, in my case - AWS account is managed by external organization and creating new VPCs are not allowed for security reasons).

Please take a look at my PR
#137

README content update

I feel like for someone needing a quickstart, there is an opportunity to add some additional clarity and documentation for getting this up and running quickly. Can the README be updated to provide some quick commands to get started on building this infra out?
Thanks.

Large zip files fail to scan

#54

[ERROR] FileNotFoundError: [Errno 2] No such file or directory: '7za'
Traceback (most recent call last):
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/metrics/metrics.py", line 184, in decorate
    response = lambda_handler(event, context)
  File "/var/lang/lib/python3.8/site-packages/aws_lambda_powertools/logging/logger.py", line 347, in decorate
    return lambda_handler(event, context)
  File "/var/task/lambda.py", line 76, in lambda_handler
    expand_if_large_archive(
  File "/var/task/lambda.py", line 142, in expand_if_large_archive
    archive_summary = subprocess.run(
  File "/var/lang/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/var/lang/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/var/lang/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)

This function is not properly handling large ZIP files.

Feature Request: add keyPattern option for addSourceBucket method

This feature would allow the user to pass an optional key pattern to this method as a second argument so that in the case that the user is using a "global" bucket (bucket used by others but has a target object within the bucket), the user can specify the pattern to the target and give permissions to only update objects on trigger that are in that target. If there was no pattern passed, then the original '*' would be used for the keyPattern.

Wrongly tagged the file

Hi,

we have tested this solution and works well.

we tested with sample EICAR file and the scan tagged the object as INFECTED.

we have added the same file to a ZIP contains some other files, the scan tagged as CLEAN. Please advice on this.

Regards,
Srinivas

npm package can't be installed with pnpm

Probably some packaging problem because pnpm tries to read some file but it's a directory, steps to reproduce:

❯ pnpm init
Wrote to /private/tmp/123/package.json

{
  "name": "123",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

❯ pnpm add cdk-serverless-clamscan


Packages: +1
+
Packages are hard linked from the content-addressable store to the virtual store.
  Content-addressable store is at: /Users/user/Library/pnpm/store/v3
  Virtual store is at:             node_modules/.pnpm
 EISDIR  EISDIR: illegal operation on a directory, read



Progress: resolved 1, reused 0, downloaded 0, added 0

❯ pnpm --version
7.25.0

How can I delete a file once its been tagged as INFECTED?

Hello!

Sorry for the silly question, but I'm not sure in CDK how I can add a rule to delete a file marked as infected? I presume this is possible?

I notice you have an infectedRule? which creates an event bridge rule? But I wasn't sure how to go about doing this.

My existing code which works now

    const BackEndBucket = S3.Bucket.fromBucketName(
      this,
      `bucketname`,
      `bucketname}`,
    );

    const sc = new ServerlessClamscan(this, 'Clamscan', {
      acceptResponsibilityForUsingImportedBucket: true,
    });

    sc.addSourceBucket(BackEndBucket);

    const bucketPolicy = new BucketPolicy(this, 'BucketPolicy', {
      bucket: BackEndBucket,
    });

    bucketPolicy.document.addStatements(sc.getPolicyStatementForBucket(BackEndBucket));

    const infectedTopic = new Topic(this, `InfectedTopic`);
    sc.infectedRule?.addTarget(
      new SnsTopic(infectedTopic, {
        message: RuleTargetInput.fromEventPath(
          '$.detail.responsePayload.message',
        ),
      }),
    );

feature : allow user to choose the EFS performance mode

Currently the EFS created by this utility uses General Performance mode. Given the reliance on EFS and file operations there should be a choice for the user to use max i/o mode for non latency sensitive use-cases.

cdk-serverless-clamscan version

2.1.26

Language

Python

Download object is forbidden 403.

Hi,
this is possibly related to #532 but not entirely sure.

After a successful cdk deploy I can see that the bucket policy is attached to the input bucket. The input bucket was set up with objectOwnership: ObjectOwnership.BUCKET_OWNER_ENFORCED.

When uploading a file to the input bucket, the tagging works. However, when the scan lambda function tries to download the file, a 403 Forbidden error is thrown.

The download works if I remove the attached bucket policy.

I was in contact with AWS support to figure out if this is an issue with our setup or with the bucket policy itself. I presume others would run into the same 403 Forbidden issue if the issue would be related to the bucket policy.
Unfortunately, the AWS support was unable to pinpoint the exact reason of the 403 Forbidden, but they helped me re-write the bucket policy so that it does the same thing (prevent downloading the file unless it's clean) and also have a working download.

According to the docs here, for the Deny and NotPrincipal combination, it is recommended that the root account is also added to the list of principals. Currently, it is not, but that did not work for me either.

Here is what is working for me, and I believe it achieves the same functionality as before.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::my-bucket",
                "arn:aws:s3:::my-bucket/*"
            ],
            "Condition": {
                "StringEquals": {
                    "s3:ExistingObjectTag/scan-status": [
                        "IN PROGRESS",
                        "INFECTED",
                        "ERROR"
                    ]
                },
                "ArnNotEquals": {
                    "aws:PrincipalArn": [
                        "arn:aws:sts::111111111111:assumed-role/my-role/my-function",
                        "arn:aws:iam::111111111111:role/my-role"
                    ]
                }
            }
        }
    ]
}

From my testing, this is not a breaking change.

I will open a PR as well. Do you see any issues that would prevent the merge of the PR?

Question regarding VPC and EFS

As far as I understand the architecture, it's only necessary to have a VPC with private subnet and S3 Gateway Endpoint because the solution is using EFS to support larger files that would exceed the Lambda runtime /tmp storage of 512MB?

I could also directly copy the files from S3 to the Lambda /tmp folder as long as my use case doesn't need more than 512MB of temp storage that the Lambda runtime provides, correct?

Thank you!

No space left on device error

version: "0.0.37"
config: basically using "Example 2. (Bring your own destinations)" from the README.

I've had several medium sized files error with LibClamAV Error: cli_writen: write error: No space left on device.

I've seen the issue with files that are .tar.gz or .zip that are several hundred MB or larger but are under the 4GB limit that would trigger the file being unzipped by the function.

For example, I uploaded a 1.3GB mp4 file which scanned successfully.

"message": {
        "source": "serverless-clamscan",
        "input_bucket": "mybucket",
        "input_key": "service-mbrs-ntscrm-02324682-02324682.mp4",
        "status": "CLEAN",
        "message": "Scanning /mnt/lambda/ba0853c7-23b3-418e-8123-02e7a2cba057/service-mbrs-ntscrm-02324682-02324682.mp4\nLibClamAV Warning: Bytecode run timed out in interpreter after 5000 opcodes\nLibClamAV Warning: Bytecode 51 failed to run: CL_ETIMEOUT: Time limit reached\nLibClamAV Warning: Bytecode run timed out in interpreter after 5000 opcodes\nLibClamAV Warning: Bytecode 51 failed to run: CL_ETIMEOUT: Time limit reached\n/mnt/lambda/ba0853c7-23b3-418e-8123-02e7a2cba057/service-mbrs-ntscrm-02324682-02324682.mp4: OK\n\n----------- SCAN SUMMARY -----------\nKnown viruses: 8543498\nEngine version: 0.103.2\nScanned directories: 1\nScanned files: 1\nInfected files: 0\nData scanned: 2884.52 MB\nData read: 1354.23 MB (ratio 2.13:1)\nTime: 678.677 sec (11 m 18 s)\nStart Date: 2021:07:07 17:31:42\nEnd Date:   2021:07:07 17:43:00\n"
    },

but when I tar and gzip that file (still 1.3GB), it throws the No space left on device error.

{
    "source": "serverless-clamscan",
    "input_bucket": "mybucket",
    "input_key": "service-mbrs-ntscrm-02324682-02324682.tar.gz",
    "status": "ERROR",
    "message": "ClamAV exited with unexpected code: 2.Scanning /mnt/lambda/798ee59f-4a86-4119-8da1-eb05a7054950/service-mbrs-ntscrm-02324682-02324682.tar.gz\nLibClamAV Error: cli_writen: write error: No space left on device\n/mnt/lambda/798ee59f-4a86-4119-8da1-eb05a7054950/service-mbrs-ntscrm-02324682-02324682.tar.gz: Can't write to file ERROR\n\n----------- SCAN SUMMARY -----------\nKnown viruses: 8543498\nEngine version: 0.103.2\nScanned directories: 1\nScanned files: 0\nInfected files: 0\nTotal errors: 1\nData scanned: 1441.06 MB\nData read: 1353.11 MB (ratio 1.06:1)\nTime: 117.868 sec (1 m 57 s)\nStart Date: 2021:07:07 17:42:57\nEnd Date:   2021:07:07 17:44:54\n"
}

Same error when I just zip the file, but also includes Warning: cli_unzip: falied to write 8192 inflated bytes:

{
    "source": "serverless-clamscan",
    "input_bucket": "mybucket",
    "input_key": "service-mbrs-ntscrm-02324682-02324682.zip",
    "status": "ERROR",
    "message": "ClamAV exited with unexpected code: 2.Scanning /mnt/lambda/e748820b-1c37-47e9-a7aa-c4084263d153/service-mbrs-ntscrm-02324682-02324682.zip\nLibClamAV Error: cli_writen: write error: No space left on device\nLibClamAV Warning: cli_unzip: falied to write 8192 inflated bytes\n/mnt/lambda/e748820b-1c37-47e9-a7aa-c4084263d153/service-mbrs-ntscrm-02324682-02324682.zip: Can't write to file ERROR\n\n----------- SCAN SUMMARY -----------\nKnown viruses: 8543498\nEngine version: 0.103.2\nScanned directories: 1\nScanned files: 0\nInfected files: 0\nTotal errors: 1\nData scanned: 1440.69 MB\nData read: 1352.76 MB (ratio 1.06:1)\nTime: 121.162 sec (2 m 1 s)\nStart Date: 2021:07:07 18:22:03\nEnd Date:   2021:07:07 18:24:04\n"
}

I tried with a bunch of other tar.gz files as well, and it seems like any file that is larger that ~300MB when uncompressed failed with the no space error, but files smaller than that scanned successfully.

I think clamscan handles zip and tar files internally by unarchiving them to a temp directory, and it is filling up the lambda VM temp directory (/tmp or /var/tmp by default) when it unzips / untars those files.

I modified the scan lambda locally, adding a tempdir in the EFS mount (with the --tempdir option) and all of the files that had previously failed now scanned successfully.

My changes are up at main...bibliotechy:add-efs-temp-dir

Happy to make a PR if you think this is the right approach.

Get [Resource handler returned message: "'MemorySize'] error when deploying

I got this error when trying deploying using cdk deploy

Resource handler returned message: "'MemorySize' value failed to satisfy constraint: Member must have value less than or equal to 3008 (Service: Lambda, Status Code: 400, Request ID: f7011447-a4f7-4771-9bbe-2eb2f900cd73)" (RequestToken: 30119af5-3151-bd09-f803-9b570eb9f61b, HandlerErrorCode: InvalidRequest)

not so sure how to fix it, as memory size is not configurable.

Any help would be appreciated

How do I use the OnResult function correctly?

Hello!
Sorry I'm back with another question, so I want to use the onResult function to send the payload to a Lambda function so I have done this in CDK.

    const clamScanDeleteLambda = new lambda.Function(this, 'clamscan-delete-file', {
      runtime: lambda.Runtime.NODEJS_14_X,  
      code: lambda.Code.fromAsset('lambda'),  
      handler: 'index.handler',  
    });

    const sc = new ServerlessClamscan(this, 'Clamscan', {
      resultDest: new destinations.LambdaDestination(clamScanDeleteLambda)
    });

However it doesn't seem to like the destination defined in this way?

Object literal may only specify known properties, and 'resultDest' does not exist in type 'ServerlessClamscanProps'.ts(2345)

Am I doing this wrong?

Can't attach bucket policy to imported Buckets that are created using CDK but made in other stacks (With proposed solution)

First of all, this project has saved me a lot of time, so thank you for that.

I'm opening this issue because I'm finding that I need to be able to control certain aspects of the Lambda Function in order to use it across other stacks that use it.

I mainly need:

  1. The ability to give the Scanning Lambda and the Scanning Lambda Role a concrete name so that I can use them to create bucket policies using CDK in imported bucket stacks with confidence.

In short:
Proposed Solution - Parameterise the Scan Lambda Name and Role Name .
Would this cause issues?

In Long:
The problem I am having without (1) is that I can't reference the lambda in other stacks where I create S3 buckets.
I have existing buckets. They are made in CDK in other stacks. There are a good amount (20ish) of buckets. This can't be changed.

I am importing those buckets into the stack that creates the ServerlessClamscan construct.

// Bucket Stack

const bucket = new S3.Bucket(this, "bucketID", "BucketA")
// Clamscan Stack
    const importedBucket = S3.Bucket.fromBucketName(this, `imported-bucket-${bucketName}`, "BucketA");

    const sc = new ServerlessClamscan(this, "s3-bucket-antivirus-scanner", {
      buckets: [importedBucket],
      acceptResponsibilityForUsingImportedBucket: true,
    });

In the bucket Stack I would like to be attach the scan policy with CDK. this would require having access to the scan function rolename and lambda name. To avoid a circular dependency I would like to pass in the Scan Lambda Name and Scan Lambda Role Name into both the ServerlessClamscan stack and the Bucket Stack. This would allow me to not have manual deploy steps

// Bucket Stack

const bucket = new S3.Bucket(this, "bucketID", "BucketA")

   
    getPolicyStatementForBucket(){
       // manually reimplement in the Bucket Stack OR export a helper function that consistently generates the correct bucket policy 
       // possible if we pass in the scan function role name and scan function name
    }
    const result = bucket.addToResourcePolicy(
        getPolicyStatementForBucket(bucket),
     );

What happens after the definition bucket is updated?

Does it scan again all the inputBuckets? Or is the scanning only triggered by fileUpload to inputBucket?

I am trying to avoid the corner case when a previously 'CLEAN' file, would be flagged as 'INFECTED' due to definition update.

Thank you !

P.S.: Perhaps this information could also be added to the README.md

CDKv2 support

Hi,
when do you plan on providing this for CDKv2 too?

deploy fails setting up bucket notifications

Hi,

I've deployed this before on my test environments, about a month ago.
Now when I try to deploy this on my prod AWS account I'm getting this error:

CdkMalwareFileCheckStack-stg | 53/57 | 12:39:53 PM | CREATE_FAILED | Custom::S3BucketNotifications | rBucket-stg/Notifications (rBucketstgNotificationsE6E8EC6D) Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configur
ations overlap. Configurations on the same bucket cannot share a common event type.. See the details in CloudWatch Log Stream:..

I've updated the CDK lib dependencies and still have the same issue

Run in VPC behind proxy

Hi,
we would love to use this library but we have some corporate restrictions.

Something like.

f.write("\nHTTPProxyServer $proxyServer")
f.write("\nHTTPProxyPort $proxyPort")
f.write("\nHTTPProxyUsername $proxyUsername")
f.write("\nHTTPProxyPassword $proxyPassword")
  • Every Lambda must run in a VPC.
    We have this covert with via an Aspect but an option to configure it would be nice to have.

Would you be open for a PR?

Lambda fails when build on Arm64

Hi, it would be nice to be able to specify either additional docker build args or the lambda architecture.
Because when deploying the cdk from e.g. a Mac M1, the python lambda will fail with

IMAGE	Launch error: fork/exec /var/lang/bin/python3: exec format error	Entrypoint: [/var/lang/bin/python3,-m,awslambdaric]	Cmd: [lambda.lambda_handler]	WorkingDir: [/var/task] | IMAGE Launch error: fork/exec /var/lang/bin/python3: exec format error Entrypoint: [/var/lang/bin/python3,-m,awslambdaric] Cmd: [lambda.lambda_handler] WorkingDir: [/var/task]

I guess because the image is build for arm64 (when run on a mac m1), but the lambda architecture is set to X86_64.

So either let's add an option to specify the architecture in FunctionProps (this would require clamav to work on arm64) or some way to add additional docker build args like --platform linux/amd64 so the image gets build for X86_64 if the build host is arm64.

I'm using cdk-serverless-clamscan v1.
Let me try with v2...

Make memory size and concurrent scans count configurable

Current Lambda function runs at maximum permitted memory size of 10GB. Coupled with cold start latency and possible bursts of concurrent invocations in response to many files uploaded at once, it might be beneficial to make amount of reserved concurrency and memory size configurable.
I understand that ClamAV can use quite a bit of memory, but it hasn't been using much more than 1.5GB only in my test cases.
Right now it's possible to reconfigure scan lambda using escape hatches, but it's quite easy to expose memory size and reserved concurrency settings as construct props.

I am willing to implement that myself if required.

The virus database is older than 7 days!

Hello,

not sure if it's an issue or I messed the point on how to update it?
refereing this issue #143

issue starting from 2023-01-17
Lambda DownloadDef doesn't report errors logs:

{
    "level": "INFO",
    "location": "decorate:440",
    "message": {
        "version": "0",
        "id": "b30047e4-83f1-6cd8-309d-b60db16d6b01",
        "detail-type": "Scheduled Event",
        "source": "aws.events",
        "account": "XXXXXX",
        "time": "2023-01-22T17:22:22Z",
        "region": "eu-west-1",
        "resources": [
            "XXX"
        ],
        "detail": {}
    },
    "timestamp": "2023-01-22 17:23:02,604+0000",
    "service": "freshclam-update",
    "cold_start": true,
    "function_name": "XXX",
    "function_memory_size": "1024",
    "function_arn": "arn:aws:lambda:eu-west-1:927280550839:function:seyna-platform-main-fileM-rClamScanDownloadDefs097-cXi4giQVVTDY",
    "function_request_id": "",
    "xray_trace_id": ""
}

I saw "Update the docker images of the Lambda functions with the latest version of ClamAV by re-running cdk deploy."
We deploy everyday but maybe I miss a way to force this update?

Scanner lambda fails with daily.cld not found

Every time the scanner lambda gets a notification about a new object to scan, it starts and then stops with an error saying it cannot find the daily.cld file in the definitions bucket.

It seems as if that file should be downloaded along with the other files by the definitions downloader lambda but no matter how many times I trigger it, it doesn't do so.

What am I missing?

LibClamAV Warning

I started seeing this warning in the message on scan results:

"LibClamAV Warning: **************************************************\nLibClamAV Warning: *** The virus database is older than 7 days! ***\nLibClamAV Warning: *** Please update it as soon as possible. ***\nLibClamAV Warning: **************************************************\n"
I re-ran cdk deploy and it updated the Lambda functions, but I continue to see that warning.

Is there another step to update the virus database and get rid of the warning message?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.