Design discussion: AWS assertions about terratest HOT 7 OPEN

Hi @otterley! 👋

Thanks for the good questions :)

Some responses:

What should the function signatures look like? Here's an idea for a pattern: AssertHas. For example, AssertEC2InstanceHasAddr(t *testing.T, instanceID string, addr string).

Good question. Have you seen analogous patterns in other testing libraries we could adopt to avoid reinventing the wheel? If not, the pattern you listed seems like a great start. The only thing I would add is that each AssertXXXHasYYY method should call a separate GetYYY method under the hood (e.g., AssertEC2InstanceHasAddr uses GetEC2InstanceHasAddr) to retrieve the info it asserts on. That will allow users to fetch that info without an assertion using the same code path.

Should assertions live in a separate repository, like terratest-assertions? (I'm thinking that the scope of this repository is already getting reasonably wide; it might improve velocity if the assertions live elsewhere.)

I always like to start with the simplest thing that could work. In this case, I think a separate package within the Terratest repo would be simpler. It would allow the code to be versioned together, tested together, documented together, etc. If it turns out that the amount of assertion code becomes unmanageable, it can always be broken out into a separate repo later.

Should assertions be struct methods? In order not to be excessively verbose, it might be convenient to wrap a *testing.T and client.ConfigProvider[1] into a struct and then call the assertion methods on it.

Probably the best way to answer this question is to write some actual code and then write the prettiest test code for it that you can think of, even if the methods / structs you're using in that test code don't yet exist (the "wishful thinking" approach from SICP). If the API you come up with works better with a struct, then by all means, let's use it. Might be worth doing this process as a PR, even with code that doesn't actually compile, so we can all discuss.

from terratest.

What should the function signatures look like?

I think that a combo of interface and struct would work nicely in this case. For example, I could imagine the following:

package assert

type EC2InstanceAssertion interface {
 HasIpAddress() bool
}

type EC2Instance struct {}

func (instance *EC2Instance) HasIpAddress(t *testing.T, instanceID string, addr string) {
  // call some method here to validate
}

Then you could use interface embedding to extend the assertions:

type ECSInstance struct {
   *EC2Instance
} 

func (ecsInstance *ECSInstance) HasAgentVersion(t *testing.T, instanceID string, version string) {
   // validate it has a particular ECS agent version
}

Then in a test...

assert.EC2Instance.HasIpAddress(t, "i-1234567890abcdef0", "10.1.10.54")

from terratest.

@mcalhoun I like where you're going with this. Taking it a perhaps a step too far...

package assert

import (
    "testing"
    "github.com/gruntwork-io/terratest/assertions/aws/ec2"
)

type assertion *testing.T

func In(t *testing.T) assertion {
    return assertion(t)
}

func (a *assertion) EC2Instance(instanceID string) ec2.Instance {
    return ec2.FindInstanceByID(a, instanceID)
}

package ec2

import (
    "testing"
    awsec2 "github.com/aws/aws-sdk-go/service/ec2"
)

type Instance struct {
    awsec2.Instance
    t *testing.T
}

func FindInstanceByID(t *testing.t, instanceID string) Instance {
    ...
)

// fail if IP address does not match
func (i *instance) HasIPAddress(addr string) {
}

Then tests can be run like so:

assert.In(t).EC2Instance("i-1234567890abcdef0").HasIPAddress("10.1.10.54")

The biggest question I have is whether, by loading up the assert package with constructors, the compiled size of a test will be extremely large, even if most of the constructors are never called. Is this a problem in practice?

from terratest.

I like the fluent API, but can't tell if it would work well for many different types of cases. I still strongly recommend using real-world examples to drive this API, rather than starting with hypotheticals :)

The biggest question I have is whether, by loading up the assert package with constructors, the compiled size of a test will be extremely large, even if most of the constructors are never called. Is this a problem in practice?

Given that this is just used for testing, I'm not sure why we'd care about compile size...

from terratest.

Ah, I was thinking about the compile time, not the size per se. Apologies for the misdirection.

from terratest.

Ah, gotcha. I haven't seen it become much of an issue in Go unless the project gets huge, but that's a problem that can be solved if we ever get there.

from terratest.

Just found this: https://onsi.github.io/gomega/

(Mainly mentioning it here as a reminder, not sure whether it's a good fit.)

from terratest.