AWS SDK selects the default credentials unless aws_profile is set in an .envrc.
# Example configurationexport AWS_PROFILE="engineering"
The credentials precedence is
Credentials set in .envrc OR as an Environment variable.
Credentials set in ~/.aws/credentials AND ~/.aws/config AND AWS_PROFILE set as an Environment variable.
Credentials set in ~/.aws/credentials AND ~/.aws/config AND AWS_PROFILE is NOT set as an Environment variable. Default credentials are used.
AWS Region
The aws_region parameter queries resources in a specific region. If not provided, the AWS region set in environment variables or configuration files are used.
Assuming an IAM role allows an IAM users gain additional (or different) permissions to perform actions in a different AWS account. (See example aws configure IAM role)
Each resource requires specific permissions to perform the operations required for testing. For example, to test an AWS EC2 instance, your service principal requires the ec2:DescribeInstances and iam:GetInstanceProfile permissions. You can find a comprehensive list of each resource's permissions needed in the documentation.
Use the Resources
Since this is an InSpec resource pack, it defines the InSpec resources and includes example tests only. To use the AWS resources in your tests, do the following:
inspec init profile --platform aws my-profile
The above command generates a sample inspec.yml that depends on master. We recommend this is pinned to a release of the resource pack as follows:
name: my-profiletitle: My own AWS profileversion: 0.1.0inspec_version: '>= 4.6.9'depends:
- name: inspec-awsurl: https://github.com/inspec/inspec-aws/archive/x.tar.gzsupports:
- platform: aws
Use the Resources
Since this is an InSpec resource pack, it only defines InSpec resources. To use these resources in your controls, create your profile:
Create a profile
inspec init profile --platform aws my-profile
The above command generates a sample inspec.yml that depends on master. We recommend this is pinned to a release of the resource pack as follows.
Example inspec.yml:
name: my-profiletitle: My own AWS profileversion: 0.1.0inspec_version: '>= 4.6.9'depends:
- name: inspec-awsurl: https://github.com/inspec/inspec-aws/archive/x.tar.gzsupports:
- platform: aws
This resource pack allows the testing of the following AWS resources. If a resource you wish to test is not listed, please feel free to open an Issue. As an open-source project, we also welcome public contributions via Pull Request.
For disallowing FTP, we check that there is no ingress from 0.0.0.0/0 on port 21. The below sample control loops across all regions, checking all security groups for the account:
title'Test AWS Security Groups Across All Regions For an Account Disallow FTP'control'aws-multi-region-security-group-ftp-1.0'doimpact1.0title'Ensure AWS Security Groups disallow FTP ingress from 0.0.0.0/0.'aws_regions.region_names.eachdo |region|
aws_security_groups(aws_region: region).group_ids.eachdo |security_group_id|
describeaws_security_group(aws_region: region,group_id: security_group_id)doit{shouldexist}it{should_notallow_in(ipv4_range: '0.0.0.0/0',port: 21)}endendendend
Test that an EC2 instance is running and using the correct AMI
To provide multi-region support, the aws_region property is specified to a resource. This property affects AWS resources that have a region dependency. For example, security groups. One special case worth mentioning is the aws_s3_bucket resource that updates its region based on the location returned from S3.
The aws_regions resource is used to loop across all regions.
aws_regions.region_names.eachdo |region|
<useregioninotherresourceshere>
end
aws_endpoint
A custom endpoint URL can optionally be specified to resources for testing other compatible providers. This propagates to the AWS client configuration. An example is provided below for Minio S3 compatible buckets.
title'Test For Minio Buckets Existing at a Custom Endpoint'endpoint=attribute(:minio_server,value: 'http://127.0.0.1:9000',description: 'The Minio server custom endpoint.')control'minio-buckets-1.0'doimpact1.0title'Ensure Minio buckets exist.'describeaws_s3_bucket(aws_endpoint: endpoint,bucket_name: 'miniobucket')doit{shouldexist}enddescribeaws_s3_bucket(aws_endpoint: endpoint,bucket_name: 'notthere')doit{should_notexist}endend
{{< note >}}
The InSpec AWS assumes full compatibility with the underlying AWS SDK, and unsupported operations cause failures. Hence, depending on the external provider implementation, your mileage may vary!
{{< /note >}}
aws_retry_limit and aws_retry_backoff
In certain cases, AWS implements rate-limiting. To mitigate this issue, the Retry Limit and Retry Backoff can be set in two ways:
1) Environment Variables
Setting AWS_RETRY_LIMIT and AWS_RETRY_BACKOFF environment variables is implemented at the session level.
export AWS_RETRY_LIMIT=5
export aws_retry_limit=5
Note environment variables are case insensitive.
2) InSpec Control
InSpec AWS resources now support setting the Retry Limit and Retry Backoff at the control level, as shown below.
InSpec AWS depends on version 3 of the AWS SDK provided via Train AWS. InSpec depends on Train AWS, so this is not explicitly listed in the Gemfile here.
Running a sample profile using Docker
A Dockerfile is provided at the root of this resource pack repository.
This test runs the unit tests, creates the target infrastructure, and runs the intergration tests. If successful, the test automatically destroy everything. If it fails, it will keep the environment up, testing then can be achieved by running:
make int_test
The AWS credentials can either be supplied via environmental variables or files located on ./aws folder.
Add a filter criterion to aws_kms_keys (plural) to filter AWS-managed keys from customer-managed keys.
π InSpec and Platform Version
2.1.21
π Possible Solutions
Note that aws_kms_key (singular) provides a managed_by_aws? matcher. I think the same name would be best. And it would likely give hints to implementation.
The way that BillingReports work in AWS is heavily tied to the 'parent' account and does not work with child accounts in linked organizations through consolidated billing. The account you use must exist in the parent billing org and cannot be a child org.
We might consider aws budgets resource, as that allows access to the billing reports for a child 'org'.
When you do the following, InSpec performs a full enumeration of AWS users, making the same API call and getting the same result.
describe aws_iam_users do
# some tests
end
describe aws_iam_users do
# some other tests
end
That is potentially slow. Can that be cached? In this case, it's the IAM client's list_users API call, but any others could be treated the same - they typically return hashlike structs.
We'd need to be able to disable the cache from the inspec CLI.
This is an ephemeral, per-process cache - not to be persisted to disk.
π InSpec and Platform Version
2.2.10
π€ Replication Case
Timing runs when the same resource is hit repeatedly should be similar to when it is hit only once.
Inspec fails to find cloudwatch alarms when the search metric has dimensions associated with it.
π InSpec and Platform Version
2.1.0
π€ Replication Case
Create a Cloudwatch Metric with at least 1 dimension.
Create an alarm on that metric.
Create and run a simple inspec test to find your alarm
describe aws_cloudwatch_alarm(
metric_name: "My_Metric_Name_with_dimension",
metric_namespace: "AWS/ECS",
) do
it {should exist}
end
Notice inspec fails to find the alarm
π Possible Solutions
The issue is that the allowed params for the aws_cloudwatch_alarm resource do not allow the dimension field to be passed to the request. This field is required (if the metric contains dimensions) for the describe_alarms_for_metric method in the AWS SDK.
Here are the allowed params for the inspec resource:
Notice that the dimensions field is required for this method. From the documentation:
:dimensions (ArrayTypes::Dimension) β The dimensions associated with the metric. If the metric has any associated dimensions, you must specify them in order for the call to succeed.
To resolve:
Add dimensions as an allowed field.
Pass that field to the query in the same way that metric name and namespace are being passed.
P.S.
I can open a new issue if you would prefer but for AWS resources specifically, why do we define the allowed params that are used for the request? The AWS SDK already has built in parameter validation so having Inspec define "allowed params" seems more restrictive than helpful. I would think that just passing the params from the user to the AWS SDK and then bubbling invalid param errors would be more robust / resilient than having to field support issues like these. Thanks :)
This issue supercedes #2592, which had a lot of other discussion. Refs #3138
Most AWS resources support tagging - attaching a (sometimes limited) number of key-value pairs. We'd like to expose this to users in several ways.
Unfortunately, the API for fetching tags varies widely by services, so each resource will need to implement the actual fetch, while a shared library might be able to handle the properties / matchers /filter criteria definitions.
Thoughts on desired behavior:
Singular resources
tags property returning a hash
Already implemented by aws_ec2_instance, PR for aws_s3_bucket on #3138
has_tag?(tag_name) matcher
has_tag_value?(tag_name, tag_value) matcher
Plural resources
tags merged hash property. Are values boosted to arrays?
Azure also defines custom properties, based on the observed tags: name_tag, env_tag, etc. Is that desirable? How do you handle absence? Is that possible to implement as filter criteria on a plural?
Case sensitivity?
AWS tags are sometimes titlecased (Name is, traditionally).
describe aws_aurora_cluster(db-cluster-identifier: 'idauto-inspec-dev20180717') do
it { should exist }
its { 'database_name'} { should eq master }
its {'status'} { should eq 'available' }
its {'db_cluster_members' { should include 'rds-idauto-aurora-inspec-dev-0' }
end
Filter would be db-cluster-identifier, and the properties returned would be AvailabilityZones, DatabaseName, Status, MultiAZ, Engine, EngineVersion and DBClusterMembers
Since InSpec is still not where awspec is today, such an access could help users maintain speed and not open issues here and there whenever we find something is not readily available.
The new AWS InSpec resources are multi-region as required e.g.
aws_regions.region_names.each do |region|
aws_security_groups(aws_region: region).group_ids.each do |security_group_id|
describe aws_security_group(aws_region: region, group_id: security_group_id) do
it { should_not allow_in(ipv4_range: '0.0.0.0/0', port: 21) }
end
end
end
As such, the concept of a 'default region' is less relevant than before. Currently we end up with lots of errors if no region is explicitly set in the environment.
π€ Replication Case
Run any profile depending on this resource pack without the region set:
$ inspec exec my-profile -t aws://
π Possible Solutions
Our current implementation requires AWS_REGION to be set but we could now reasonably add a default to us-east-1. This would require copying the SDK region detection and injecting us-east-1 if nothing is found.
AWS S3 allows configuring lifecycle policy on S3 buckets for its objects. Lifecycle policy allows either moving the objects to a different bucket tier or deleting them after a certain period of time. We can have a compliance check in inspec if the lifecycle policy is set and verify the validity time of the lifecycle
I'm probably missing a simple step somewhere. Based on your docs it suggests we can run tests against more filters than the examples documented in your user docs.
Your docs also mention needing permissions ec2:DescribeRouteTables suggesting you're making api calls to ec2.describe-route-tables
If you are making calls into ec2.describe-route-tables then we should be able to do more than verify a route table exists and what the route table id is.
Something like the following should be possible:
describe aws_route_tables do
its ('vpc_ids') { should include 'a_vpc_id' }
its ('route.nat_gateway_id') { should exist }
its ('route.nat_gateway_id') { should include 'a_nat_gateway_id' }
end
Potentially, this a π feature request
As mentioned above, something like the following should be possible:
describe aws_route_tables do
its ('vpc_ids') { should include 'a_vpc_id' }
its ('route.nat_gateway_id') { should exist }
its ('route.nat_gateway_id') { should include 'a_nat_gateway_id' }
end
AWS S3 buckets can be configured to activate versioning
This is very important to avoid accidental suppression of content by overwriting files, as well as for traceability/security (esp. when combined with MFA deletion)
This feature request is to have a special matcher have_versioning_enabled implemented for the aws_s3_bucket resource along with the existing have_access_logging_enabled or have_default_encryption_enabled, so one could do:
describe aws_s3_bucket('test_bucket') do
it { should exist }
it { should have_versioning_enabled }
end
AWS IAM (managed) policies may be either customer-managed or AWS-managed. AWS provides about 370 managed policies. If you are only interested in one or the other, there is a lot of paging to do.
The API supports fetching one or the other, as well as both by default.
π InSpec and Platform Version
2.1.43
π Possible Solutions
Add something like managed_by_customer and managed_by_aws properties / criteria.
for 'aws_security_groups' as done for 'aws_security_group'.
Something like this:
all_groups = aws_security_groups
describe all_groups.where(vpc_id: 'vpc-00120123') do
it { should allow_in(ipv4_range: "0.0.0.0/0", port: 80)}
it { should_not allow_in(ipv4_range: "0.0.0.0/0", port: 22)}
end
This will be useful if the instance has several security groups to abstracting on behalf of the group id, group name and etc. It will be helpful for infrastructure tests. Thanks.. :-)
A contribution from a community member, #3504, uses new features such as modules-as-resource-bundles, local grouping, and new Terraform versions. Much of our AWS (and possibly Azure) test fixture code could likely be simplified by adopting similar practices.
From discussion on #3576, it is suspected that the count and position of AWS security group rules is not round-trip-stable.
If it is the case that the rule count and position is unstable, using the position criterion with allow_*_only is likely unstable and dangerous.
It should be noted that position was never intended to reflect the order-of-evaluation by AWS for security groups; it merely was a tool to isolate a particular rule for examination.
π InSpec and Platform Version
3.0.28
π€ Replication Case
Example: create a security group with one rule that targets IP Range A, and another rule that targets IP Range B, with no other criteria. describe_security_groups may now return a structure that reflects only one rule, but contains a list of IP address ranges (in other words, the two rules were combined into one, with the same semantics).
It is suspected this also occurs for security_group_id targets, as well; that criterion is introduced on #3576.
π Possible Solutions
Issue deprecations for position
Use the description field on rules to allow the user to isolate a rule
Other ideas welcome
Tread carefully not to break semantics of allow_*_only (assuming those are widely understood, which is dubious).
π I'm requesting the addition of a tags resource property with aws_rds_instance, as it's also available with aws_ec2_instance. The motivation is that we're moving from awspec to InSpec at a large client who needs to have all resources properly tagged. Many thanks!
The aws-s3-buckets resource does not capture AccessDeniedorInvalidAccessKeyId errors.
Incase of any of the above errors, the resource wrongly reports an empty bucket_names array without an exception.
π InSpec and Platform Version
inSpec 3.0
π€ Replication Case
Run inspec aws-s3-buckets resource with Invalid keys int he ENV
inspec> aws_s3_buckets.bucket_names
=> []
Expception reported by AWS CLI
aws s3 ls
An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: The AWS Access Key Id you provided does not exist in our records.
π Possible Solutions
Capture AccessDenied and InvalidAccessKeyId in the below location.
Line 55:
# Intercept AWS exceptions
def catch_aws_errors
yield
rescue Aws::Errors::MissingCredentialsError
# The AWS error here is unhelpful:
# "unable to sign request without credentials set"
Inspec::Log.error "It appears that you have not set your AWS credentials. You may set them using environment variables, or using the 'aws://region/aws_credentials_profile' target. See https://www.inspec.io/docs/reference/platforms for details."
fail_resource('No AWS credentials available')
rescue Aws::Errors::ServiceError => e
fail_resource e.message
end
end
When using the aws_route_table resource, there is a vpc_id attr_reader in the code but when you put it in a control like:
control "aws_route_table::RouteTable" do
impact 1.0
describe aws_route_table("rtb-34ca3e4f") do
it { should exist }
its("vpc_id") { should eq "vpc-58342a21" }
end
end
you get
Γ aws_route_table::RouteTable: Iggy bjc-demo-aws-a2.json aws_route_table::RouteTable (1 failed)
β Route Table rtb-34ca3e4f should exist
Γ Route Table rtb-34ca3e4f vpc_id should eq "vpc-58342a21"
expected: "vpc-58342a21"
got: nil
(compared using ==)
Would like a aws_nat_gateway and aws_nat_gateways resource.
π Possible Solutions
describe aws_nat_gateway(nat-gateway-id: nat-1da12de71e7fee8db) do
it { should exist }
end
describe aws_nat_gateway.where(vpc_id: vpc-123456 ) do
its('count') { should eq 2 }
its('statee) { should cmp ' }
end
Properties would be, vpc_id, nat-gateway-id and state.
When the accessing user has insufficient permission to call GetAccountSummary, various aws_iam_root_user tests fail without indicating it is a permission issue.
Expected: failure, but with a message indicating a missing permission.
π InSpec and Platform Version
2.1.60
π€ Replication Case
Here, inspec-aws-test-default-no-perms is an AWS profile referencing a user with literally no permissions.
[cwolfe@lodi inspec]$ be inspec shell -t aws://us-east-2/inspec-aws-test-default-no-perms
Welcome to the interactive InSpec Shell
To find out how to use it, type: help
You are currently running on:
Name: aws
Families: cloud, api
Release: aws-sdk-v2.11.33
inspec> describe aws_iam_root_user do
inspec> it { should have_mfa_enabled }
inspec> end
Profile: inspec-shell
Version: (not specified)
AWS Root-User
Γ should have mfa enabled
expected #has_mfa_enabled? to return true, got false
Test Summary: 0 successful, 1 failure, 0 skipped
π Possible Solutions
This is a tough one. have_mfa_enabled is boolean. Exception-throwing is constrained here, as well.
When running tests on an AWS account without an IAM Password Policy the user is presented with the following error:
Γ iam_password_policy-expiration: Passwords must be set to expire
Γ IAM Password-Policy should expire passwords
undefined method `expire_passwords' for nil:NilClass
InSpec and Platform Version
Platform: Arch Linux
InSpec: 2.0.4
Possible Solutions
Add handling for @policy being nil and/or raise ResourceSkipped/ResourceFailed.
I execute a scan without an AWS target. I receive a stack trace.
$ inspec exec training-aws-security
/opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/resources/aws/aws_vpc.rb:20:in `block (2 levels) in <class:AwsVpc>': undefined method `[]' for nil:NilClass (NoMethodError)
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/resources/aws/aws_vpc.rb:15:in `to_s'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core/metadata.rb:180:in `build_description_from'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core/metadata.rb:133:in `populate'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core/metadata.rb:258:in `create'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:421:in `set_it_up'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:385:in `subclass'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/rspec-core-3.7.1/lib/rspec/core/example_group.rb:259:in `block in define_example_group_method'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner_rspec.rb:27:in `example_group'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:304:in `rspec_skipped_block'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:265:in `get_check_example'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:281:in `block in register_rule'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:280:in `each'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:280:in `flat_map'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:280:in `register_rule'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:97:in `block in load'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:96:in `each'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:96:in `load'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/runner.rb:103:in `run'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/lib/inspec/cli.rb:168:in `exec'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
from /opt/inspec/embedded/lib/ruby/gems/2.4.0/gems/inspec-2.0.17/bin/inspec:12:in `<top (required)>'
from /opt/inspec/bin/inspec:72:in `load'
from /opt/inspec/bin/inspec:72:in `<main>'
π I'm requesting the addition of a tags resource property with aws_security_group, as it's also available with aws_ec2_instance. The motivation is that we're moving from awspec to InSpec at a large client who needs to have all resources properly tagged. Many thanks!
aws_iam_user has a diverse set of properties and matchers, each of which requires a separate API call to fetch things like the login profile (which contains password rotation info), access keys, MFA device information, and with #2719, information about inline and attached policies (which are separate API calls). All of this is fetched at InSpec resource creation (in fetch_from_api), regardless of whether it is ever accessed.
Most or all of those should be fetched on-demand.
π InSpec and Platform Version
2.1.33
π Possible Solutions
Several other singular resources do lazy fetching.
describe aws_iam_role(IAM_MASTER_ROLE_NAME) do
it { should exist }
its('inline-policies') { should include IAM_MASTER_INLINE_POLICY }
its('attached-policies') { should include IAM_MASTER_ATTACHED_POLICY }
end
List inline policies aws iam list-role-policies --role-name <role_name>
List attached policlies aws iam list-attached-role-policies --role-name <rolename>
All plural resources are expected to provide a where method to allow filtering (at least that expectation is there if the resource exposes any fields, which aws_iam_policies does).
The docs also do not treat the resource in the typical way for a plural resource, like including info about filter criteria.
π InSpec and Platform Version
2.1.43
π€ Replication Case
[cwolfe@lodi inspec]$ bundle exec inspec shell -t aws://us-east-2/inspec-aws-test-default-readonly
Welcome to the interactive InSpec Shell
To find out how to use it, type: help
You are currently running on:
Name: aws
Families: cloud, api
Release: aws-sdk-v2.11.33
inspec> p = aws_iam_policies
=> IAM Policies
inspec> p.entries.count
=> 370
inspec> p.arns.count
=> 370
inspec> p.where { arn.include?('aws') }
NoMethodError: undefined method `where' for IAM Policies:#<Class:0x00007f9bdf077218>
from (pry):4:in `load_with_context'
inspec> exit
[cwolfe@lodi inspec]$ date; git log -n1 --oneline
Fri Apr 13 10:52:52 EDT 2018
3355f682 (HEAD -> master, origin/master, origin/HEAD) Update CHANGELOG.md to reflect the promotion of 2.1.43
π Possible Solutions
Just add filter.add_accessor as per usual; and update docs.
inspec exec ${CLOUD}/test/controls/vpc.rb -t aws://my-region/my-profile
Internally inspec uses the ruby aws-sdk which defines the http(s) endpoints for the API calls.
π InSpec and Platform Version
2.2.10
π€ Replication Case
We are using Stratoscale:
[symphony]
access_key = ""
secret_key = ""
endpoint_url = "https:///api/v2/aws/ec2/"
π I'm requesting the addition of a tags resource property with aws_s3_bucket, as it's also available with aws_ec2_instance. The motivation is that we're moving from awspec to InSpec at a large client who needs to have all resources properly tagged. Many thanks!
π I'm requesting the addition of a tags resource property with aws_vpc, as it's also available with aws_ec2_instance. The motivation is that we're moving from awspec to InSpec at a large client who needs to have all resources properly tagged. Many thanks!