mypy errors for CloudFormation custom resource types about aws-lambda-typing HOT 8 CLOSED

Hi @chuckwondo, thanks for your contribution.

The structure you see is TypedDict inheritance and is valid. It seems mypy is interpreting it wrong, and there is an issue reported.

from aws-lambda-typing.

Hi @MousaZeidBaker,

My pleasure to help. I very much appreciate that you've created this repo! And so do my colleagues who I just shared this with.

Thanks for pointing me to that mypy issue. The thread for that issue makes sense.

However, even if that weren't an issue, I'd suggest that the refactoring I've proposed (and submitted a PR for), is still a worthwhile adjustment, for at least the following reasons:

• The Create event type is missing
• Each specific event type should be constrained to its respective, single Literal value. For example, the Delete event type always has a ResultType of "Delete". It will never have a ResultType of "Create" or "Update". Similarly for the others.
• The "generic" Event type is not a "merging" of the 3 specific types, it is a Union of them

from aws-lambda-typing.

Glad to hear that :)

Actually the create event is indirect there, its the common class that represents it since the create event doesn't have anything unique. With that said, I think the adjustments you made add some value, however, there are some issues that needs to be solved.

1. The first is to add a docstring to the variable CloudFormationCustomResourceEvent.
   

2. The CloudFormationCustomResourceEvent type doesn't work anymore, if you replace the specific types (create, update, delete) in the tests to CloudFormationCustomResourceEvent mypy will fail and I think this one will most likely be used, not even sure the specific types needs to be exported (unless I'm missing something).
   

from aws-lambda-typing.

Regarding the Create type, I think having the "common" type represent the Create type is not ideal, as it's not obvious.

Regarding the typing error in the previous screenshots, that's to be expected. From the "producer" end of things, you'll use the specific types when producing an event. On the "consumer" end (i.e., as an event handler), you'll use the generic type, and then use the value of the "tag" (RequestType) to narrow the type.

For example:

def handle_event(event: CloudFormationCustomResourceEvent) -> None:
    if event["RequestType"] == "Update":
        # event is automatically narrowed to CloudFormationCustomResourceUpdate
        old_properties = event["OldResourceProperties"]

See mypy's documentation on tagged unions.

from aws-lambda-typing.

I've updated the PR with a couple more tweaks, and a new test that uses the generic event type, showing automatic type narrowing.

from aws-lambda-typing.

Regarding the Create type, I think having the "common" type represent the Create type is not ideal, as it's not obvious.

Agree!

Regarding the typing error in the previous screenshots, that's to be expected. From the "producer" end of things, you'll use the specific types when producing an event. On the "consumer" end (i.e., as an event handler), you'll use the generic type, and then use the value of the "tag" (RequestType) to narrow the type.

I understand this, and I was talking from a consumer point of view because as you already mention they will use CloudFormationCustomResourceEvent in the handler and it failed previously, but it seems that they work now after your latest changes. Thats also why I in the tests use the types that the consumer will use the handlers.

I still think we should add the previous docstring to the CloudFormationCustomResourceEvent variable, because as you can see the current docstring when you hover over it is not so helpful.

from aws-lambda-typing.

Also, now that mypy src works, can you add the command as an extra sanity check to the workflows: here, here, and here.

from aws-lambda-typing.

Cool. I appreciate your prompt replies, and I'll work on your latest change requests.

Removing total=False is what allows things to work like you show in your latest screenshot.

Also, just to clarify a bit more on the producer/consumer topic, I consider your pre-existing test functions to be "producer" functions because you are producing an object (as a literal dict expression) in each one (even though you don't actually do anything with it, such as pass it to a "consuming" function). These functions are representative of what occurs on the CloudFormation side, where the events are produced.

For example, I consider the following a "producer" function, which is why it doesn't really make sense to type event generally as CloudFormationCustomResourceEvent. In this case, since we are producing (creating) the object, we know it's specific type, because we're producing it, so we use the specific type:

def test_cloud_formation_custom_resource_create_event() -> None:
    event: CloudFormationCustomResourceCreate = {
        "RequestType": "Create",
        "RequestId": "unique id for this create request",
        "ResponseURL": "pre-signed-url-for-create-response",
        "ResourceType": "Custom::MyCustomResourceType",
        "LogicalResourceId": "name of resource in template",
        "StackId": "arn:aws:cloudformation:us-east-2:123456789012:stack/mystack-sggfrhxhum7w/f449b250-b969-11e0-a185-5081d0136786",  # noqa: E501
        "ResourceProperties": {
            "key1": "new-string",
            "key2": ["new-list"],
            "key3": {"key4": "new-map"},
        },
    }

That's why I added an addition test function to act as a "consumer", where we don't know in advance the specific type, so we use the "general" type, and use the "tag" (the RequestType property in our case) to discern the specific type:

def test_handle_cloud_formation_custom_resource_event(
    event: CloudFormationCustomResourceEvent,
) -> None:
    if event["RequestType"] == "Update":
        # event is automatically narrowed to CloudFormationCustomResourceUpdate
        old_properties = event["OldResourceProperties"]

That way, these are more representative of "real" code, since the producer functions are actually on the CloudFormation side (i.e., they're not functions we would actually write because CloudFormation is the producer of the events), and the consumer function is what we would write to handle (consume) events appropriately.

from aws-lambda-typing.