iacsecurity / tool-compare Goto Github PK

Pull test cases from https://github.com/bridgecrewio/checkov/blob/master/tests/terraform/checks/resource/aws that are not yet covered by existing test cases in this repo.

Most of these checks are simpler. Bridgecrew and Indeni announced support for context-aware rules. Would be good to have more test cases that support context-type issues.

FYI @christophetd

I was looking at the ebs optimized check and noticed that the instance type used has EBS as optimized. the instances that support but don't have default are;

c1.xlarge
c3.xlarge
c3.2xlarge
c3.4xlarge
g2.2xlarge
i2.xlarge
i2.2xlarge
i2.4xlarge
m1.large
m1.xlarge
m2.2xlarge
m2.4xlarge
m3.xlarge
m3.2xlarge
r3.xlarge
r3.2xlarge
r3.4xlarge

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-optimized.html

• Formatting of the table is too tight (can't see tfsec for example) + sub-headers are too small
• Add a README in each directory inside test-cases to see in plain english what does this code even trying to do and test in general terms.
• Add versions and dates (last execution)

But it's still in beta status.
Can I just change Custom Rule Support - Snyk cell into "yes" ?

https://snyk.io/blog/developing-custom-iac-rules-with-snyk-iac/
https://docs.snyk.io/products/snyk-infrastructure-as-code/custom-rules

Can you please re-run checks for checkov. There are some updates on checks.
Also that would be great if you can add Last Scan time to the readme to see when it is executed.

https://github.com/iacsecurity/tool-compare/blob/main/test-cases/terraform/aws/best-practices/rds_retention_period_set/main.tf

This is related to rds_retention_period_set

the main.tf specifies to check preferred_backup_window

resource "aws_rds_cluster" "default" {
  cluster_identifier      = "aurora-cluster-demo"
  engine                  = "aurora-mysql"
  engine_version          = "5.7.mysql_aurora.2.03.2"
  availability_zones      = ["us-west-2a", "us-west-2b", "us-west-2c"]
  database_name           = "mydb"
  master_username         = "foo"
  master_password         = "bar"
  preferred_backup_window = "07:00-09:00"
}

should not it check backup_retention_period ?

resource "aws_rds_cluster" "rds_cluster_2" {
  cluster_identifier      = "demo"
  engine                  = "aurora-postgresql"
  availability_zones      = ["us-west-2a", "us-west-2b", "us-west-2c"]
  database_name           = "mydb"
  master_username         = "foo"
  master_password         = "bar"
  backup_retention_period = 5
  preferred_backup_window = "07:00-09:00"
  enabled_cloudwatch_logs_exports = ["audit", "error"]
}

I may be wrong, pls correct me.

Following #7 , need to re-run tfsec tests and check for additional cases caught.

looks like some of the tools are being executed on the "plan" phase while others directly on the IaC files?
Is the comparison been done only for terraform? Some of the tools support multiple IaC

Over the past year, infrastructure-as-code security began to evolve to look beyond just the code itself. We've seen this with Accurics, Bridgecrew, Fugue and Indeni Cloudrail's offerings. We also saw this recently with driftctl's launch.

An IaC security tool comparison needs to take this into account. This means we need to create a staging AWS environment that can be used in conjunction with the IaC security scans to show the capabilities beyond just static analysis.

We should figure out a way to pull this information into this tool-compare repository in a manner that's reproducible to anyone who seeks to do so.

Its great that you have this repo and have samples and provide failing code, but also show what would pass also?

IAM and other test cases.

Thanks for putting together a nice set of tools comparison. Have some basic doubts

For tag_all_items, the support is provided by only Indeni Cloudrail and Kics

Hope the test case is executed against the provided main.tf file and the result is shared, pls correct me

One of the below confusion, while checking for the test-cases/terraform/aws/best-practices/tag_all_items/ , the checkov_results.txt contains

• Check: CKV_AWS_26: "Ensure all data stored in the SNS topic is encrypted"
• Check: CKV_AWS_27: "Ensure all data stored in the SQS queue is encrypted"

In the case of a check, the checkov_results.txt does not indicate any pass/fail, the test cases CKV_AWS_26 & CKV_AWS_27 are totally different checks on AWS, in this context should this be included as part of the test case folder?

Additionally,
Does this indicate we need to just check if all resources provisioned in TF are associated with tags? some of the resources are not supported tags, how do we handle them by this validation?

@christophetd maybe something you want to take on, but fugue's Regula is missing in this list.
https://github.com/fugue/regula

Tools have various capabilities, beyond just catching test cases. Would like to add a table in the README that provides this information, beyond just the licensing line we have there now.

From an implementation perspective, this will be a YAML file that is used to auto-generate the README.

Thanks (:

Time to start adding more test cases for other CSPs and IaC languages. Ahead of that, I'd like to clean up the structure a bit, run an updated execution (been a couple of months since the last execution) and add Azure test cases (still Terraform).