k1low / awspec Goto Github PK

Hi @k1LoW!
I always use awspec!

When '--cluster' is not specified.
containerInstances is nothing.

➜  ~ aws ecs describe-container-instances --container-instances 11111111-2222-3333-4444-555555555555
{
    "failures": [
        {
            "reason": "MISSING",
            "arn": "arn:aws:ecs:ap-northeast-1:000000000000:container-instance/11111111-2222-3333-4444-555555555555"
        }
    ],
    "containerInstances": []
}

https://github.com/matsuzj/awspec/blob/master/spec/type/ecs_container_instance_spec.rb#L5-L14
Can you only set container-instance ID?

describe ecs_container_instance('my-container-instance') do
  it { should exist }
  it { should be_active }
  it { should_not be_inactive }
  its(:status) { should eq 'ACTIVE' }
  its(:container_instance_arn) do
    should eq 'arn:aws:ecs:us-east-1:123456789012:container-instance'\
              '/f2756532-8f13-4d53-87c9-aed50dc94cd7'
  end
end

https://github.com/matsuzj/awspec/blob/master/lib/awspec/helper/finder/ecs.rb#L21-L24
Maybe, There are two arguments.

def find_ecs_container_instances(cluster_name, container_instances)
  res = ecs_client.describe_container_instances(cluster: cluster_name, container_instances: container_instances)
  res.container_instances if res.container_instances
end

I ran into issue - if there are AWS access/secret keys defined as env. variables (e.g. AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY) then they overwrite the values defined in spec/secrets.yml.

It would be great to update Readme.md to note this fact.

Regards,
Constantin

Currently the only way to specify the profile we're working with is via the AWS_PROFILE environment variable. I think this capability should also be reflected in the secrets.yml file.

Version: 0.85.2

I generated a series of tests using the generate command. They all worked, with the exception of those that are commented out below (note that one 'all' test that's passing for sg-22222222)

describe security_group('SG_TestTest02') do
  it { should exist }
  its(:group_id) { should eq 'sg-11111111' }
  its(:group_name) { should eq 'SG_TestTest02' }
  its(:inbound) { should be_opened(1433).protocol('tcp').for('x.x.x.x/32') }
  its(:inbound) { should be_opened(80).protocol('tcp').for('x.x.x.x/16') }
  its(:inbound) { should be_opened(80).protocol('tcp').for('x.x.x.x/32') }
  its(:inbound) { should be_opened.protocol('all').for('x.x.x.x/24') }
  its(:inbound) { should be_opened.protocol('all').for('x.x.x.x/24') }
  # these indeed exist, but there seems to be a bug with awspec at least at version 0.85.2
#  its(:inbound) { should be_opened.protocol('all').for('sg-33333333') }
#  its(:inbound) { should be_opened.protocol('all').for('sg-44444444') }
  its(:inbound) { should be_opened.protocol('all').for('sg-22222222') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/16') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/16') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/20') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }
#  its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }
  its(:outbound) { should be_opened.protocol('all').for('0.0.0.0/0') }
  its(:inbound_rule_count) { should eq 17 }
  its(:outbound_rule_count) { should eq 1 }
  its(:inbound_permissions_count) { should eq 4 }
  its(:outbound_permissions_count) { should eq 1 }
  it { should belong_to_vpc('vpc - vpc') }
end

This generates the same error for each failing test:

Failure/Error: its(:inbound) { should be_opened(443).protocol('tcp').for('x.x.x.x/32') }

      NoMethodError:
        undefined method `group_name' for nil:NilClass
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:106:in `block in cidr_opened?'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:98:in `select'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:98:in `cidr_opened?'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:26:in `block in inbound_opened?'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:25:in `each'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:25:in `find'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:25:in `inbound_opened?'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/type/security_group.rb:15:in `opened?'
      # /usr/local/bundle/gems/awspec-0.85.2/lib/awspec/matcher/be_opened.rb:3:in `block (2 levels) in <top (required)>'
      # ./spec/us-west-2_spec.rb:130:in `block (2 levels) in <top (required)>'

Will testing of VPC endpoints and related route table routes be included at some point?

I came across this whilst testing routes within a route table which contained an endpoint. The generate command shows:

it { should have_route('').target(gateway: 'vpce-xxxxxxxx') }

but the same spec gives an error

Failures:
  1) route_table 'my_route_table' should have route ""
     Failure/Error: it { should have_route('').target(gateway:'vpce-xxxxxxxx') }
       expected route_table 'my_route_table' to have route ""

Thanks

It would be nice to be able to be more flexible in tests. For example, I usually setup my VPCs with multiple types of subnets which have controls like whether or not they have public IP addresses, what other subnets they're allowed to talk to, etc with a subnet of each class in each VPC of a region.

So, if I have a class of subnets that has only internal IPs and I don't care which one an instance is in as long as it is in one of those, I might write a test with a list of those subnet ids like so:

  describe ec2('i-1234567') do
    let(:valid_subnets) { ['subnet-1234567', 'subnet-4567890'] }
    context 'instance definition' do
      it { should exist }
      it { should belong_to_vpc("MYVPC") }
      it { should belong_to_subnet(valid_subnets) }
    end
  end

Or, I might just name them all similarly, like "MYVPC Private Subnet in AZ A", such that I could then try to match based on tag:Name like so:

  describe ec2('i-1234567') do
    let(:vpc_name) { 'MYVPC' }
    let(:valid_subnets) { vpc_name + ' Private*' }
    context 'instance definition' do
      it { should exist }
      it { should belong_to_vpc(vpc_name) }
      it { should belong_to_subnet(valid_subnets) }
    end
  end

Is there a way to test ALB and related resources with awspec as it is? Or does this need to be added as a new set of types?

Hi,
Given the following spec:

describe vpc('main-vpc') do
  it { should exist }
  it { should have_tag('Environment').value('development') }
end

There follows the output below:

vpc 'main-vpc'
  should exist
  should have tag "Environment"

Shouldn't the message be "should have tag value "development""?

Version of awspec used is 0.61.1

Currently the have_policy test will only pass if you specify the randomly generated ids for Policy Id, Statement Id and IAM user Id in the spec. This means you first have to obtain these ids (which I'm currently getting via the sdk). I understand the def in /lib/awspec/type/s3_bucket.rb is just comparing json, but for future, it would be nice to exclude these ids from the comparison.

    {
        "Version": "2012-10-17",
        "Id": "#{policy_id}",
        "Statement": [
                {
                        "Sid": "#{statement_id}",
                        "Effect": "Allow",
                        "Principal": {
                                "AWS": "#{aws}"
                        },
                        "Action": "s3:PutObject",
                        "Resource": "arn:aws:s3:::#{bucket_name}/AccessLogs/AWSLogs/#{ac_id}/*"
                }
        ]
    }

    POLICY
  end

I made such spec file named 'test_spec.rb'.

describe ec2('test-instance') do
  it { should be_running }
end

then I executed 'rake spec', I got error such as.

/Users/tomokazu.hirai/Documents/infra_ci/awspec/work/spec/test_spec.rb:8:in `<top (required)>': undefined method `ec2' for main:Object (NoMethodError)
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1361:in `load'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1361:in `block in load_spec_files'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1359:in `each'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/configuration.rb:1359:in `load_spec_files'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:102:in `setup'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:88:in `run'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:73:in `run'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/lib/rspec/core/runner.rb:41:in `invoke'
        from /Users/tomokazu.hirai/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/rspec-core-3.4.1/exe/rspec:4:in `<main>'

Any idea do you have ?

The iam_role resource documentation suggests using the following syntax:

describe iam_role('my-iam-role') do
  its('attached_policies.count') { should eq 5 }
end

Doing so results in the following error being raised:

Awspec::BlackListForwardable::CalledMethodInBlackList:
  Method call :attached_policies is black-listed
# .../gems/awspec-0.60.0/lib/awspec/resource_reader.rb:9:in `method_missing_via_black_list'
# .../gems/awspec-0.60.0/lib/awspec/type/base.rb:62:in `method_missing'

The alternate suggested form ("resource.attached_policies.count") gives the same error.

It doesn't look like this has been implemented. I attempted to add code that would search by name but it doesn't appear to be working. I'd be happy to do this work but I need a little guidance. Should I only have to modify the find_network_interface in the ec2 finder or is there something else I have to update to get this working?

      def find_network_interface(interface_id)
         res = ec2_client.describe_network_interfaces({
                                                          filters: [
                                                            {
                                                              name: 'network-interface-id',
                                                              values: [interface_id]
                                                            }
                                                          ]
                                                        })
        resource = res.subnets.single_resource(interface_id)
        return resource if resource
        res = ec2_client.describe_network_interfaces({
                                                 filters: [{ name: 'tag:Name', values: [interface_id] }]
                                               })
        res.network_interfaces.single_resource(interface_id)
      end

When I run:

$ awspec generate security_group "vpc-11cd8628"

the generated output doesn't include the rules with other security group ids:

describe security_group('dev-sg') do
  it { should exist }
  its(:group_id) { should eq 'sg-13b4588b' }
  its(:group_name) { should eq 'dev-sg' }
  its(:inbound) { should be_opened(80).protocol('tcp').for('99.238.255.50/32') }
  its(:inbound) { should be_opened(8080).protocol('tcp').for('99.238.255.50/32') }
  its(:inbound) { should be_opened }
  its(:inbound) { should be_opened(22).protocol('tcp').for('62.19.60.171/32') }
  its(:inbound) { should be_opened(22).protocol('tcp').for('99.238.255.50/32') }
  its(:inbound) { should be_opened(5986).protocol('tcp').for('62.19.60.171/32') }
  its(:inbound) { should be_opened(3306).protocol('tcp').for('99.238.255.50/32') }
  its(:outbound) { should be_opened(80).protocol('tcp').for('0.0.0.0/0') }
  its(:outbound) { should be_opened }
  its(:outbound) { should be_opened(123).protocol('udp').for('0.0.0.0/0') }
  its(:outbound) { should be_opened(443).protocol('tcp').for('0.0.0.0/0') }
  its(:inbound_rule_count) { should eq 7 }
  its(:outbound_rule_count) { should eq 4 }
  its(:inbound_permissions_count) { should eq 6 }
  its(:outbound_permissions_count) { should eq 4 }
  it { should belong_to_vpc('vpc1') }
end

Is this an expected behavour?

Regards,
Constantin

In the attached_to? method for InternetGateway, test for state == 'attached' is wrong. The state should be available.

Funny tough that the official AWS ruby SDK is inconsistent in the documentation.

In one place it says

resp.internet_gateways[0].attachments[0].state #=> String, one of "attaching", "attached", "detaching", "detached"

In another place it says

attachment.state - The current state of the attachment between the gateway and the VPC (available). Present only if a VPC is attached.

I've tested it and it only returns available.

I have a spec like:

    context 'The bastion AMI'do
      subject { ami("ami-b282d1d2") }
      it { should have_tag('state').value('good') }
    end

When I run it, I get:

  1) The bastion The bastion AMI should have tag "state"
         On host `35.177.250.135'
         Failure/Error: it { should have_tag('state').value('good') }
         ArgumentError:
           wrong number of arguments (given 3, expected 1)

         # /usr/lib64/ruby/gems/2.3.0/gems/awspec-0.69.0/lib/awspec/type/base.rb:56:in `method_missing'
         # /usr/lib64/ruby/gems/2.3.0/gems/awspec-0.69.0/lib/awspec/matcher/have_tag.rb:4:in `block (2 levels) in <top (required)>'
         # /usr/lib64/ruby/gems/2.3.0/gems/rspec-expectations-3.5.0/lib/rspec/matchers/dsl.rb:70:in `block (2 levels) in match'
         # /usr/lib64/ruby/gems/2.3.0/gems/rspec-support-3.5.0/lib/rspec/support.rb:103:in `with_failure_notifier'

It looks like this can be fixed with a change to lib/awspec/type/ami.rb, to add tags_allowed to line 4:

module Awspec::Type
  class Ami < ResourceBase
    aws_resource Aws::EC2::Image
    tags_allowed

This is consistent with the Ec2 type.

I'll submit a pull request shortly.

Hi all,

recently I started using awspec for my IaC projects. However I'm facing an issue with iam_role, be_allowed_action and IAM inline policies.

Awspec is not able to determine the permissions and allowed AWS actions if the attached policy is an inline policy. If I attach a managed policy to the IAM role, the be_allowed_action works perfectly. It's that the normal behaviour? or is a bug? For my tests it would be amazing if awspec would be able to determine the allowed actions when I use an inline policy.

Those are my tests:

`describe iam_role('Access_Role') do
it { should exist }
end

describe iam_role('Access_Role') do
it { should be_allowed_action('s3:PutObject').resource_arn('arn:aws:s3:::bucket-staging/') }
it { should be_allowed_action('s3:GetObject').resource_arn('arn:aws:s3:::bucket-staging/') }
it { should be_allowed_action('s3:ListBucket').resource_arn('arn:aws:s3:::bucket-staging/*') }
end`

And this is the result:

`iam_role 'Access_Role'
should exist

iam_role 'Access_Role'
should be allowed action "s3:PutObject" (FAILED - 1)
should be allowed action "s3:GetObject" (FAILED - 2)
should be allowed action "s3:ListBucket" (FAILED - 3)

iam_role 'BOS_App_Access_Role'
should have inline policy "Access_Role_Inline_Policy"
should have inline policy "Access_Role_Inline_Policy"
`
As you can see, the specific tests for inline policy works fine, while allowed actions not. And I'm absolutely sure the buckets have those permissions as I tested manually.

Thank you guys.

Hi. Looks like the way JRuby computes module names messes with this code here:

https://github.com/k1LoW/awspec/blob/master/lib/awspec/helper/finder.rb#L55

"#{Aws::Lambda::Client}" comes out as "Module:0x12345::Client"

Tried hacking that a bit to get beyond (forcing the right name), but other exceptions followed. Giving up at this point.

You may not care if it runs on JRuby :) but thought I'd give this issue a go.

I do have a situation where I'd like to run awspec, but don't have access to a Ruby runtime, and I do have access to a JVM.

Hi,

When checking the existence of a VPN connection using the vpn_connection resource, it fails to return/find anything if the name is specified rather than the ID.

  describe vpn_connection("sandbox-vpn-dc2") do
    it { should exist }
  end

begin exception VPN vpn_connection 'sandbox-vpn-dc2' should exist expected vpn_connection 'sandbox-atcloud-dc2' to exist end exception
    should exist (FAILED - 2)

If I do the same however using the VPN ID, it successfully finds the existence of the VPN:

  describe vpn_connection("vpn-abc123") do
    it { should exist }
  end

 vpn_connection 'vpn-abc123'
    should exist

If there's any useful debug information I can provide please let me know, I'm not all too familiar with this!

Environment

aws-sdk (2.9.44)
awspec (0.81.1)
rspec (3.6.0)

At the moment there is no spec support for internet gateway so those cannot be verified. It would be nice if support was added for it.

AWS Reference doc: http://docs.aws.amazon.com/sdkforruby/api/Aws/EC2/InternetGateway.html

Hi,

Provided I have a hosted zone foo, with a record bar, the following spec will return an error, contradicting what the doc says:

describe route53_hosted_zone('foo.') do
  it { should have_record_set('bar.foo.') }
end

============

route53_hosted_zone 'foo.'
  should have record set "bar.foo." (FAILED - 1)

Failures:

  1) route53_hosted_zone 'foo.' should have record set "bar.foo."
     Failure/Error: it { should have_record_set('bar.foo.') }

     TypeError:
       no implicit conversion of nil into String
     # ./spec/route53_spec.rb:2:in `block (2 levels) in <top (required)>'

Trying out the resources_type examples for Cloudfront Distribution

My Spec

.....
it { should exist }
it { should be_deployed }
its(:http_version) { should eq '2' }
its(:comment) { should eq "My CF Distribution." }
its(:enabled) { should eq true }
.....

Results

should exist
should be deployed
http_version
should eq "2" (FAILED - 2)
comment
should eq "My CF Distribution."
enabled
should eq true
..
2) cloudfront_distribution 'XXXXXXXXXX' http_version should eq "2"
Failure/Error: should eq '2'

   expected: "2"
        got: nil

   (compared using ==)

Looking through lib/awspec/type/cloudfront_distribution.rb - not sure where this mapping takes place?

I ran into this issue while building out stacks with terraform, where I'm frequently destroying/recreating entire ec2 clusters. If you terminate an instance, then create a new identical instance (same name), awspec seems to test against the terminated instance rather than the running one. IMO if two instances exist with the same name tag, one running and one terminated, it should assume the running one.

Hello @k1LoW

do you think is possible add a security_group_count to ec2 in order to test the number of SG attached to an EC2 instance. As today I need to validate the name of security group attached and also the number, in order to not miss eventual SG manually attached to EC2.

Something like:

describe ec2('TEST EC2') do
   it { should exist }
   it { should be_running }
   it { should have_security_group('TEST SG') }
   its(:security_group_count) { should eq 1 }
end

Thank you.
Simone

Currently awspec does not support testing of security group outbound rules to endpoints. This is similar to issue #214 which was fixed to allow testing of routes to VPC endpoints in route tables.

We have a security group egress rule

HTTPS TCP 443 pl-xxxxxxxx

which fails when I try to test it

its(:outbound) { should be_opened('443').protocol('tcp').for('pl-xxxxxxxx') }

and does not show as an entry in awspec generate.

ecs_cluster's have_container_instance() behavior is unlike most of the other matchers like it.

For example:

describe launch_configuration("foo") do
    it { should have_security_group('sg-123') } # passes with message: should have security group "sg-123"

Is not like:

  describe ecs_cluster("bar") do
    it { should have_container_instance('foobar') } # fails
    it { have_container_instance('foobar') } # passes with unhelpful message: example at ./test/ecs_cluster_spec.rb:9

The same pattern that launch_configuration uses is repeated in multiple locations and seems like have_container_instance matcher should behave the same way. Instead, however, if you use the should syntax, your test fails with the following error:

undefined local variable or method `arn_or_uuid' for ecs_cluster 'foobar':Awspec::Type::EcsCluster

Hi @k1LoW!

Love AWS Spec and would love to use it even more for IAM testing. Specifically, I am looking to test for the absence of IAM Inline Policies.

So, instead of testing this:

it { should_not have_inline_policy('AllowS3BucketAccess') }

I'd love to be able to to test something like this:

it { should_not have_inline_policy }

The idea here is that IAM users with inline policies might be bad (depending on your org's compliance guidelines).

Any thoughts?

The awspec to test whether a network acl has an associated subnet takes a very long time. We have around 50 subnets and looking at awspec/type/network_acl.rb and awspec/helper/finder/subnet.rb it seems to be looping through all the subnets each time it calls the has_subnet? definition, which is what is taking the time. It would be good to find a way to change this to speed it up.

All other awspec tests run much faster. We have ~950 examples which finish in around 2 minutes 40 seconds.

The NACL tests alone with 59 examples takes ~8 minutes.

describe network_acl(expect_nacl_id) do
  it { should exist }
  it { should belong_to_vpc(expect_vpc_name) }

  subnets.each do | expected_subnet |
    it { should have_subnet(expected_subnet) }
  end
end

Hi,

It would be nice to have have_tag support for ebs volume too.
This is useful for testing the storage costs are allocated properly in billing report.

Regards,
Constantin

Hi, when there are two AWS resources with the same name but belonging to different entities, awspec fails the tests.

For example, assume there are 2 VPCs with 2 Security Groups:

vpc-1 
vpc-2

my-security-group => vpc-1
my-security-group => vpc-2

The following test will fail:

describe security_group('my-security-group') do
  it { should exist }
end

I'm not sure how I would approach this.

1. Should awspec say something "there is a name conflict, as there are two resources with the same name"?
2. Should there be another way to target a specific resource when there is another one with the same name?

Is it possible to give the EC2 resource type something other than an instance ID to allow it to match? As an example:

describe ec2(i-a1b2c3d4) do
   ...
end

works fine, but:

describe ec2(foo) do
   ...
end

doesn't.

Given that instances often don't stay around for very long, and even if they do, the instance IDs aren't how (I figure) many people refer to instances (most people I know refer using tags), is it possible to use the Name tag value in place of the instance ID to get a match on the specific resource?

Hi,

I have the resource below declared. It incorrectly returns an error on versioning.

describe s3_bucket('X') do
  it { should exist }
  it { should have_versioning_enabled }
end

Error: expected #has_versioning_enabled? to return true, got false

When I query via the command line:

aws s3api get-bucket-versioning --bucket X
{
    "Status": "Enabled",
    "MFADelete": "Disabled"
}

If I query via the Ruby SDKv2

require 'aws-sdk'
bucket_name = 'X'
s3 = Aws::S3::Resource.new(region: 'us-west-1')
bucket.versioning.status
=> "Enabled"

when testing the acl_owner attribute of an s3 bucket, it is assumed that the API will return a display_name. This only works for older AWS regions, and does not work for newer ones such as eu-west-2, eu-central-1.

As such, it is impossible to test the acl owner of a bucket in these regions. Perhaps it needs to be possible to test on ID as well as display name?

I want to be able to search for an ASG by tag name.

Hi,

It seems that the advanced example given here https://github.com/k1LoW/awspec/blob/master/doc/resource_types.md#iam_user doesn't work properly.

describe iam_user('every_user') do
    it { should exist }
    ...
    its('resource.login_profile.password_reset_required') { should eq false }
end

error:

iam_user 'every_user'
  should exist
  ....
  resource.login_profile.password_reset_required
    example at ./spec/infra_spec.rb:153 (FAILED - 1)

Failures:

  1) iam_user 'every_user' resource.login_profile.password_reset_required
     Failure/Error: its('resource.login_profile.password_reset_required') { should eq false }

     Awspec::BlackListForwardable::CalledMethodInBlackList:
       Method call :password_reset_required is black-listed
     # /usr/local/rvm/gems/ruby-2.1.8/gems/awspec-0.49.0/lib/awspec/resource_reader.rb:8:in `method_missing_via_black_list'
     # /usr/local/rvm/gems/ruby-2.1.8/gems/awspec-0.49.0/lib/awspec/resource_reader.rb:43:in `method_missing'
     # ./spec/infra_spec.rb:153:in `block (2 levels) in <top (required)>'

Finished in 2.6 seconds (files took 3.62 seconds to load)
96 examples, 1 failure

Failed examples:

rspec ./spec/infra_spec.rb:153 # iam_user 'every_user' resource.login_profile.password_reset_required

and here is the result from AWS CLI:

$ aws iam get-login-profile --user-name every_user
{
    "LoginProfile": {
        "UserName": "every_user",
        "CreateDate": "2016-05-24T12:25:00Z",
        "PasswordResetRequired": false
    }
}

Any thoughts?

Regards,
Constantin

Would you upgrade to AWS SDK 3 as that has now been released?

Happy to update the Gemspec and give a PR if you want but with it being a one-line code change I wasn't sure if it was worth it 😄

hi

we have a record named 'bar.foo.internal.' in a hosted zone named 'foo.internal.'
and its record have a 'A record' with value '10.220.1.240' and ttl 300.

then i wrote test_spec.rb such as

describe route53_hosted_zone('foo.internal.') do
    it { should exist }
    it { should have_record_set('bar.foo.internal.').a('10.220.1.240').ttl(300) }
end

and i got these errors.

route53_hosted_zone 'foo.internal.'
  should exist
  should have record set "bar.foo.internal." (FAILED - 1)

Failures:

  1) route53_hosted_zone 'foo.internal.' should have record set "bar.foo.internal."
     Failure/Error: it { should have_record_set('bar.foo.internal.').a('10.220.1.240').ttl(300) }
       expected route53_hosted_zone 'bar.internal.' to have record set "bar.foo.internal."
     # ./spec/test_spec.rb:18:in `block (2 levels) in <top (required)>'

Finished in 0.02945 seconds (files took 2.62 seconds to load)
2 examples, 1 failure

Failed examples:

rspec ./spec/test_spec.rb:18 # route53_hosted_zone 'foo.internal.' should have record set "bar.foo.internal."

do i mistake to write test_spec.rb ?

I am testing the users and the count of users in a group. I am able to test that a certain policy is applied to a group. In further testing I am finding have_iam_user is returning the following:

The group with name "GROUPNAME" cannot be found

Also users.count is unable to find the group

• awspec (0.74.1)

Ns record test, Sometimes it succeeds or fails.

• Result of executing 100 test

  57 6 examples, 0 failures
  43 6 examples, 1 failure

Perhaps the cause is that there is no order guarantee of aws sdk return values,
but , I think that awspec should always return the same result.
(Other record types might also happen with 1recNval)

I propose that controlling result order in awspec.
For example,

$ diff lib/awspec/type/route53_hosted_zone.rb.org lib/awspec/type/route53_hosted_zone.rb
22c22,26
<           v = record_set.resource_records.map { |r| r.value }.join("\n")
---
>           #v = record_set.resource_records.map { |r| r.value }.join("\n")
>           r_sort = record_set.resource_records.map { |r| r.value }
>           r_sort.sort!
>           v = r_sort.map { |r| r }.join("\n")

Is there another way to always make it successful?

If there are two server instances with the same Name tag (even if one is terminated), the following code fails with a stack trace:

describe ec2('My Server') do
  it { should be_running }
end

The trace is:

1) private_instance ec2 'My Server' should be running
     On host `10.0.2.225'
     Failure/Error: it { should be_running }
     NoMethodError:
       undefined method `state' for nil:NilClass

     # /var/lib/gems/2.3.0/gems/awspec-0.61.0/lib/awspec/type/ec2.rb:29:in `block (2 levels) in <class:Ec2>'
     # /var/lib/gems/2.3.0/gems/serverspec-2.37.2/lib/serverspec/matcher/be_running.rb:7:in `block (2 levels) in <top (required)>'
     # ./spec/gocd_server/private_instance_spec.rb:9:in `block (3 levels) in <top (required)>'

Instead, Awspec::DuplicatedResourceTypeErro should be raised.

I added the sample spec from the README into the spec directory. I then run rake spec from the app root and get no output. If I run it with --trace I get:

** Invoke spec (first_time, not_needed)

What am I missing? Is it actually running the test.

Hi,

It seems that security_group has no support for tags, e.g. have_tag.
Is it possible to add it ?

Regards,
Constantin

Looks like resource matchers are not shared across all resource types.

Every resource matcher should be available to use in a resource type whenever the resource type supports that. Please see below.

# This will not work. But in AWS you can attach a tag to an ELB..
describe elb("some-elb") do
   it { should have_tag("Environment").value("development") }
end

# This will work as normal
describe autoscaling_group("some-asg") do
    it { should have_tag("Environment").value("development") }
end

As you will see, there's inconsistency of resource matchers. It would be nice to bring consistency to those as this creates confusion and prevent proper testing of AWS resources.

Many thanks.

https://aws.amazon.com/jp/blogs/aws/new-network-load-balancer-effortless-scaling-to-millions-of-requests-per-second/

http://docs.aws.amazon.com/sdkforruby/api/Aws/ElasticLoadBalancingV2/Client.html#describe_load_balancers-instance_method

awspec is often used in conjuntion with terraform. Terraform allows us to create AWS infrastructures with code and it stores its variables in a tfvars file. It would be great if we could import that file into the *spec file and use its contents as variables from there on out.

Example of tfvars file:

myEc2 = "myServer"
myAmi = "ami-b9b394ca"

Example of what *spec file would look like:

require terraform.tfvars

describe ec2(myEc2) do
  it { should exist }
  it { should be_running }
  its(:image_id) { should eq myAmi }
  its(:instance_type) { should eq 't2.micro' }
end

I'm not a ruby dev so I'm not sure this can be done, so I leave it to your consideration. Thanks.

We sometimes create our resource names dynamically with a prefix, would it be possible to define resources with Regex?

An example would be an autoscaling group prefixed with autoscaling-app.

This will enable verifying s3 bucket lifecycle rules.

It'd be good to be able to verify that a particular bucket has the required set of tags.

I was trying to run awspec tests on Jenkins where the AWS region was not set. The error I receive is the following:

Failures:

  1) alb 'app_load_balancer' should be active
     Failure/Error: it { is_expected.to be_active }
     
     NoMethodError:
       undefined method `state' for nil:NilClass
     # ./.bundle/ruby/2.1.0/gems/awspec-0.85.2/lib/awspec/type/alb.rb:17:in `block (2 levels) in <class:Alb>'
     # ./spec/application_load_balancer_spec.rb:4:in `block (2 levels) in <top (required)>'

This caused us to lose a lot of time where if awspec where a readable error would help a lot.

Does this make sense?

@k1LoW
I've been having some intermittent issues running tests. Ultimately, seems to be that AWS is throttling API calls, which is confirmed by their support...

I was wondering if there's a way to build a mechanism no AWSpec to work around this issue... i.e. handing throttling errors and not failing tests...

I can implement this and submit a pull request, no problem... Just would like to discuss it first.