Error emails upon adding users about aws-iam-identity-center-extensions HOT 6 CLOSED

Hi @allquixotic ,

Your assumption about why we are listening on user and group events is correct. This is to enable the self-sustaining function of the solution, where in you could have a '.sso' link file with a group/user that's not yet created, and later when that user/group is created , then we wanted to automatically provision that .sso link created earlier.

We've validated the solution with a dev Okta tenant through SCIM in a much earlier release, so what you are seeing suggests there may be a regression with SCIM user import.

I will debug by setting up 3.0.1 in my local dev and see where this is going wrong.

from aws-iam-identity-center-extensions.

Hi @allquixotic , could you please help provide some more data with regards to the issue you are seeing. Using solution version 3.0.1, I just stood up a dev Okta tenant , changed my SSO config from local store to external identity provider, had automatic provisioning enabled through SCIM and pushed through a user. There are no assignments pre-created for this user.
And, as expected, the userHandler lambda is triggered, then it looks up DDB and resolves empty related assignments and stops the process.

• Are you getting the error notification in the same flow?
• Could you also share these logs please so that we can get more insights into what's going on - in your target account, under cloudwatch log groups - navigate to the log group named /aws/lambda/env-ssoUserHandler. Logs for one error user creation trigger should suffice.

Thank you,
Leela

from aws-iam-identity-center-extensions.

Hi @allquixotic ,

Many thanks for the very detailed debug info, this really helped me understand what's going on and even identify a new issue.

For context, the reason you are seeing the exception happen is because the SSO user and group creation event bridge rules we deploy as part of the solution are looking for any Create type events and not currently checking on the status of this event. So, the fix would be to update the event bridge rule definitions to only listen on successful user and group creation events only.

In summary, with the really thorough analysis you've provided along with an incorrect behaviour of a use case flow we've noticed separately, we have identified 3 issues that need to be fixed here:

• Update event bridge rule definitions for all SSO and Org events the solution listens, to ensure only successful events are processed : this is handled by #45
• Triage and work on the Converting circular structure to JSON error unhandled exception: this is being tracked in #44
• Fix the asynchronous exception handling logic so that error notifications provide useful information rather than being empty or JSON object blobs : this is being tracked in #43

Please do let us know if you have any further questions/suggestions on this topic

from aws-iam-identity-center-extensions.

Hi Leela,

I think this is being caused by me issuing SCIM API calls requesting to create a group that already exists. I have written a script that pulls the relevant groups and users from our Managed AD instance and issues SCIM calls to sync them over to SSO. To avoid race conditions, my script always asks for the groups to be created, even if they already exist, because a "GET / process / POST" loop introduces the possibility of stale data. And if I do a "POST" to create a group that already exists, and get an error, my client can safely ignore the error without worry.

The reason I was getting confused about this happening during user creation is that my script creates both users and groups via SCIM. I am not sure, but you might also want to add appropriate checks in the code for events representing duplicate user creation attempts via SCIM.

For my test AD, I am creating 14 users assigned variously to a subset of 7 groups.

When I run my script, sometimes I get errors in Cloudwatch for both ssoUserHandler and ssoGroupHandler...

Here is the error from aws/lambda/env-ssoUserHandler. Note, this does not happen every time I create a user. It only seems to happen if I'm trying to add a user (via SCIM) that already exists.

2022-01-28T13:47:38.561Z	redacted-guid	ERROR	Invoke Error 	
{
    "errorType": "TypeError",
    "errorMessage": "Converting circular structure to JSON\n    --> starting at object with constructor 'IncomingMessage'\n    |     property 'req' -> object with constructor 'ClientRequest'\n    --- property 'res' closes the circle",
    "stack": [
        "TypeError: Converting circular structure to JSON",
        "    --> starting at object with constructor 'IncomingMessage'",
        "    |     property 'req' -> object with constructor 'ClientRequest'",
        "    --- property 'res' closes the circle",
        "    at JSON.stringify (<anonymous>)",
        "    at Runtime.ye [as handler] (/var/task/index.js:1:5734)",
        "    at processTicksAndRejections (internal/process/task_queues.js:95:5)"
    ]
}

Here is the error from aws/lambda/env-ssoGroupHandler:

{
    "handler": "groupsHandler",
    "logMode": "error",
    "requestId": "redacted-guid",
    "status": "FailedWithException",
    "statusMessage": "Groups operation - failed with exception: {} for eventDetail: [object Object]"
}

These errors aren't very descriptive, but hopefully the underlying cause will help you duplicate the problem :)

Issue a request like POST https://scim.us-east-1.amazonaws.com/{tenant_id}/scim/v2/Users with a payload of a user that already exists, or

POST https://scim.us-east-1.amazonaws.com/{tenant_id}/scim/v2/Groups with a payload of a group that already exists. 100% reproducible. :)

from aws-iam-identity-center-extensions.

Since we have fully triaged this, and the instant issue is resolved, I will close this issue. Will be tracking the status of the other issues to see how they turn out.

Thanks a lot for the excellent collaboration on this.

p.s. - I will re-open this issue if, after updating to 3.0.2, I still get the error emails.

from aws-iam-identity-center-extensions.

p.s. - I will re-open this issue if, after updating to 3.0.2, I still get the error emails.

Please do @allquixotic . For reference, I did execute both the duplicate user & group flows through SCIM route based on your suggestion and validated that the event bridge rules do not trigger, while the happy path routes still work with valid user & group objects. So, if you still get error emails then that would be a scenario that we did not yet handle and appreciate if you could let us know.

from aws-iam-identity-center-extensions.