The Duo stack template is mis-configuring the values for 'RADIUS server DNS name or IP addresses' in Directory Services MFA config. In one attempt, it chose an older value from CFN-DuoServiceIps- parameter set. In another it chose the latest which was set to 'default'. The resolution was to locate the proper IP values, and edit the parameter set accordingly which then triggers the lambda function to update MFA config.

Parameters

Resulting config

Hello to any and all people who are still here. I got an email stating that botocore is going to be deprecated. is there anything I need to do for this stack? Any direction would be great towards a noob like me =)

Confirmed that the Stack deployed correctly; State Manager Associations are all successful, MFA is shown as Complete in Managed AD and the RADIUS Server IP addresses match the deployed Instances.
Security Group rules all have 1812 inbound & outbound to all SG's and the VPC, and multiple users and IAM delegated roles have been created. These previously worked just entering in the user name & password, now with MFA enabled I get an error like this:
"Your user name or password can't be processed. You may also need to specify a domain name in the username (such as domain\username). If you don't know your domain name, contact your administrator."
I have disabled and re-enabled Application Access URL as well as AWS Management Console login. I have also removed and re-added different AD Users and have tried logging in with both the Short Name & full Domain DNS Name without luck.
Is there a misconfiguration on the DUO side?

When following the post-deployment steps to set up User syncing in Duo, I don't see a way that this could work.

The documentation states that we should use the Fargate's tasks IP addresses in Duo's setup, but there's no way Duo can ever reach the task because it is deployed in private subnets with a NAT Gateway, thus inbound traffic is not allowed. No NLB/ALB are created with this stack, so the Fargate Task is completely unreachable from the internet.

How should this be addressed so Duo can sync users with AWS Managed AD?

When quickstart template is run, the EC2 instances never check into SSM to be configured.

I found this is due to the EC2 instances not being able resolve DNS.

Outbound DNS is not configured in the DuoRadiusProxySecurityGroup and thus not allowed out of the EC2 instances.

I suggest either allowing DNS outbound to all hosts. Or better yet, allowing outbound to only th discovered directory services group/IPs.

Documenting a feature request for AD Sync support on behalf of a customer.

Please add a 👍 to this comment if you're interested in this feature.

When I kick the template off and the SSM document runs to pull the latest Duo security config, it errors out. I tried running the runbook separately and the same error occurs (See below). When i go into session manager and run it manually step by step it works fine.

We've already verified that AWS Directory and Radius have two-way communication and we are able to see the "fakeusername" AccessRejects on both Radius servers. Even still the MFA enabling is Failed.

The ecs Fargate services fails to start with the below error.

ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve secret from asm: service call has been retried 5 time(s): failed to fetch secret arn:aws:secretsmanager:us-east-1:......

Hello,

Setup the quickstart, but cannot get to the last stage.
Using ADConnector to O365 which works correctly
Want to setup Duo for usage in Workspaces.
Duo MFA subscription, so paid one.

cloudwatchlog UpdateDirectoryServiceMfaSettings shows
ATTEMPT 1: Creating
.....
** ATTEMPT 30: Creating
END RequestId

cloudlog d/authproxy.log remains empty
bucket duo-mfa-for-aws-director-radiusproxybootstrapsyst remains empty

RADIUS proxies have internet access via NAT in pubsubnet

opened all ports between proxies and AD connector, but no success

Can you assist in setting it up?

gr
Sander van Gemert
Sabaas

We have been using this cloud formation template for some time. Recently, we found that the solution stopped working because the Radius proxy server fails to install on the new ec2 instances. Including the following snipt from the strerr logs from one of the instances as reported to the S3 bucket:

sm/x86_64-gcc.c:606:6: error: conflicting types for 'bn_sqr_comba4'
 void bn_sqr_comba4(BN_ULONG *r, const BN_ULONG *a)
      ^
In file included from asm/x86_64-gcc.c:1:0:
asm/../bn_lcl.h:513:6: note: previous declaration of 'bn_sqr_comba4' was here
 void bn_sqr_comba4(BN_ULONG *r, const BN_ULONG *a);
      ^
make[4]: *** [x86_64-gcc.o] Error 1
make[3]: *** [subdirs] Error 1
make[2]: *** [build_crypto] Error 1
make[1]: *** [install] Error 2
make: *** [/tmp/duoauthproxy-3.2.1--src/duoauthproxy-build/usr/local/openssl/lib/libcrypto.so.1.0.0] Error 2
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 36: ./install: No such file or directory
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 56: /opt/duoauthproxy/conf/authproxy.cfg: No such file or directory
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 86: /opt/duoauthproxy/conf/authproxy.cfg: No such file or directory
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 87: /opt/duoauthproxy/conf/authproxy.cfg: No such file or directory
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 86: /opt/duoauthproxy/conf/authproxy.cfg: No such file or directory
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 87: /opt/duoauthproxy/conf/authproxy.cfg: No such file or directory
/var/lib/amazon/ssm/i-xxxxxxxxxxxxxxxxx/document/orchestration/288cc388-350a-4705-9da0-7045b71d205d/InstallAndConfigureDuoProxyService/InstallProxyService/_script.sh: line 95: /opt/duoauthproxy/bin/authproxyctl: No such file or directory

This might have something to do withthe AMI perhaps? I could be totally off base, but when I check the value of that ASM parameter, I get the following:

➜  ~ aws ssm get-parameter --name /aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2
{
    "Parameter": {
        "Name": "/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2",
        "Type": "String",
        "Value": "ami-00eb20669e0990cb4",
        "Version": 12,
        "LastModifiedDate": 1567628952.41,
        "ARN": "arn:aws:ssm:us-east-1::parameter/aws/service/ami-amazon-linux-latest/amzn-ami-hvm-x86_64-gp2"
    }
}

However, when I tried to reproduce the issue using an AmazonLinux2 AMI by running the bootstrap script manually, I found that the installation worked. Then I realized the AMI I got by running the instance from the GUI was different: ami-00068cd7555f543d5. I changed the Cloudformation template to use that AMI and everything worked just fine.

This is baffling because it doesn't look like there were any changes to that AMI recently. Perhaps the Duo Proxy Service was updated? Either way, just sending a heads up if it's helpful - perhaps it's operator error.

Thank you!

Hi,

Not enabling when trying to configure MFA DUO Software following procedure https://aws-quickstart.s3.amazonaws.com/quickstart-duo-mfa/doc/duo-mfa-for-aws-directory-service-on-the -aws-cloud.pdf

Run CloudFormantion as per documentation https://fwd.aws/nXE9E successfully.

However, when checking the RADIUS status in Directory Service, it does not activate.

Analyzing the logs via CloudWatch lambda function UpdateDirectoryServiceMfaSettings-d-id generates the following message Found addresses: [], but two are required.

Would you be able to fix this issue on the deployment.

The quickstart CloudFoundation yaml errors in existing vpc with:

Waiter ImageScanComplete failed: An error occurred (ScanNotFoundException): Image scan does not exist for the image with '{imageDigest:'null', imageTag:'scan'}' in the repository with name 'duo-authproxy' in the registry with id

Hi Team,

Based on this manual - https://aws-quickstart.github.io/quickstart-duo-mfa/, we tried to toubleshoot MFA "failed" status.

In the log - UpdateDirectoryServiceMfaSettings - e.g.
/aws/lambda/Duo-MFA-1012-UpdateDirectoryServiceMfaSettings-asF0qFKKPrSs

It returned "Radius updated successfully" after the first 6 creating attempts, then the MFA status in directory service became "completed", but subsequently it became "failed" after 3 creating attempts, and they're in the same chain. below is the snapshot of the log.

Could you please check this? any support will be appreciated.

Hi,

The "UpdateDirectoryServiceMfaSettings" Lambda failed as it tried to update the directory service while the directory services was running a backup which was confirmed by AWS. The issue we have is that we were unable to rerun the script as it looks like it's triggered from the proxy instances with data.

The MFA is used for the client VPN, the VPN was down for a week until the instances ran the RADIUS rotation script again.

My assumption is that the update script updates the proxies then updates the directory service.

Would it be possible to have a way of manually rerunning the various lambda functions and could it also be setup so that the if the directory update fails the proxies don't get updated so that they can still authenticate.

Thanks,
Chris

Currently there are no parameter support for the Directory Service MFA timeout and retries, and the defaulted values are too low for our use case.

Is there any documentation to show us what to change to support custom values?

We have used this QuickStart successfully several times in the past, as recently as April 2020.
However, as of Jan 2021 it now fails to correctly deploy because the EC2 instances do not complete the bootstrapping phase in SSM.
The "Duo-RadiusProxyBootstrapDocument" SSM command fails in the third step (ConfigureCloudWatchLogsAgent) with an example error message below. I sshed into the instance and found that the missing configuration file it is looking for and can't find is actually at the path /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm__DuoRadiusConfiguration_d-XXXXXXXXXX_CloudWatchAgentConfiguration (e.g. without the .tmp extension).
Surprisingly, the error does not cause the SSM command to fail, so the QuickStart appears to deploy correctly, but the EC2 instances are not configured with the Duo RADIUS proxy software and do not write any logs to the CloudWatch Logs group.
As a result, the RADIUS integration with the directory also fails.

The stage that is failing uses the "AmazonCloudWatch-ManageAgent" document, so perhaps there has been a change to that document between April 2020 (when we last successfully ran the QuickStart) and now?

/opt/aws/amazon-cloudwatch-agent/bin/config-downloader --output-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --download-source ssm:/DuoRadiusConfiguration/d-XXXXXXXXXX/CloudWatchAgentConfiguration --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default

Region: us-east-1

credsConfig: map[]

Successfully fetched the config and saved in /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm__DuoRadiusConfiguration_d-XXXXXXXXX_CloudWatchAgentConfiguration.tmp

Start configuration validation...

/opt/aws/amazon-cloudwatch-agent/bin/config-translator --input /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json --input-dir /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d --output /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml --mode ec2 --config /opt/aws/amazon-cloudwatch-agent/etc/common-config.toml --multi-config default

Valid Json input schema.

I! Detecting runasuser...

No csm configuration found.

No metric configuration found.

Configuration validation first phase succeeded

/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent -schematest -config /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.toml

Configuration validation second phase succeeded

Configuration validation succeeded

amazon-cloudwatch-agent start/running, process 2833

----------ERROR-------

2021/01/07 15:56:35 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.d/ssm__DuoRadiusConfiguration_d-XXXXXXXXXX_CloudWatchAgentConfiguration.tmp ...

Python 2.7 is EOL in January 2020. We need to upgrade the CopyZips function prior to then.

If the managed AD service is in maintenance mode while secrets are updating, it can lead to the service locking out everyone until the secrets are rotated successfully again. See Also: Case ID 7216850251.

The secret is stored in AWS Secrets Manager and rotated every 7 days.

Rotation is implemented by a Lambda function which generates the new secret and then it calls the SSM Run Document.

This Document updates the DUO AuthProxy configuration by updating the radius_secret_1 and radius_secret_2 settings with the new secret. After the instances are updated, an SNS message is sent to the SNS topic.

Messages sent to the SNS topic trigger the Lambda function UpdateDirectoryServiceMfaSettings-ds-instanceID which updates the Shared secret code setting in the Directory Service’s MFA settings.

If the directory is in maintenance mode, the lambda errors and does not complete. This causes the radius servers to hold the new key while the directory server holds the old key. This causes everyone to be locked out. This condition has happened to us twice now when the directory service was either being patched or snapshotting.

As a work around, we had to manually rotate the keys. Feature/bug request, check that the directory service is not under maintenance before updating the proxy hosts with new secrets? or on error backout changes? something like that.

Updated most of the arn values to reflect OSU and made good progress, however the CF is failling during the LambdaInvokePermission:

The provided principal was invalid. Please check the principal and try again. (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 4d3b7d0b-4a6e-49e5-8d49-eb00c08ecbb4; Proxy: null)

The following resource(s) failed to create: [InstanceRole, InstallAndConfigureDuoProxyServiceDocument, LambdaInvokePermission]. Rollback requested by user.

Looking for any advice!

The stack has deployed correctly and the EC2 instances started automatically and I can access them through SSM, but the Radius status is failed. When I try it again from the console. It fails agin not sure why.

CloudWatch Logs Insights
region: eu-west-1
log-group-names: /aws/lambda/UpdateDirectoryServiceMfaSettings-d-9367050d2c
start-time: -3600s
end-time: 0s
query-string:

fields @timestamp, @message
| sort @timestamp desc
| limit 20

Hello Team,

The document which walks us through this configuration process is old. I see there have been quite a few changes made related to Directory Services console, Systems Manager console with respect to the terms mentioned in this document. For Example, Step 3, point 5 states "Choose the Instances tab." but the new console has the "Resources" tab instead. There are many such terms which are different now in newer versions of console.

[+] Walkthrough document: https://aws-quickstart.s3.amazonaws.com/quickstart-duo-mfa/doc/duo-mfa-for-aws-directory-service-on-the-aws-cloud.pdf

Hi,

This solution doesn't enable MFA on directory.

• I have implemented on Ec2
• Reviewing lambda function UpdateDirectoryServiceMfaSettings-d-id, reviewing its logs found that it is not moving forward because of this error
• Found addresses: ["10.1.1.89"], but two are required.

Would you be able to fix this issue on the deployment.

related exactly to #1

Found addresses: [], but two are required.

From the lambda function.

authproxy log is empty
there are no other useful information in the logs and in the troubleshooting guide.

I've deleted and recreated this template 4 times now with the same result.

looking to figure out how to configure Duo for a client AWS workspace log in.

cloudformation is showing errors and then rolls back.
Error is GetDirectoryServiceDetails | CREATE_FAILED

I followed AWS docs and videos, worked with their support and they could not correct. advised me to go here

any help would be appreciated

When deploying aws-quickstart/quickstart-duo-mfa/scripts/packages/code_commit.zip appears to no longer be available in S3.

The DuoCleanupFunction resouce looks to have no error handling when performaing the s3.copy_object causing the next dependent resource DuoEcrCodeCommitRepo to fail due to the absense of the object.

Copying the source from another of our deployments results in success.

State Manager : Success
RADIUS Instance : Launch successfully
#Instances is 1 (Only one RADIUS Server)

When I navigate to Directory Service console the Multi-factor authentication tab is still blank. (Not even trying to set up RADIUS server)

Error log from the Lambda:
"Found addresses: [], but two are required."

edit after debug:

Lambda line #134:
#{"Name": "tag:RadiusConfigured", "Values": ["True"]},
this returns zero for some-reason

Thanks

Will this work if we are using ADFS to federate authentication to multiple applications (AWS, third-party VPN provider, etc)?

The following statement in the release isn't clear to me It says you configure your own MFA and then goes on to suggest that Duo MFA configured with this method will work..

If you use a federation mechanism like AWS Single Sign-On (AWS SSO) or Active Directory Federation Services (AD FS) with a Directory Service option, you configure your own MFA. Using Duo MFA, you log in to the AWS Management Console, and then use Duo authentication methods including Duo Push through Duo Mobile, and your Active Directory credentials to authenticate to AWS.

Hi,

Not enabling when trying to configure MFA DUO Software following procedure https://aws-quickstart.s3.amazonaws.com/quickstart-duo-mfa/doc/duo-mfa-for-aws-directory-service-on-the -aws-cloud.pdf

• Run CloudFormantion as per documentation https://fwd.aws/nXE9E successfully.

• However, when checking the RADIUS status in Directory Service, it does not activate.

• Analyzing the logs via CloudWatch lambda function UpdateDirectoryServiceMfaSettings-d-id generates the following message Found addresses: [], but two are required.

Would you be able to fix this issue on the deployment.

Solution doesn't enable MFA on specified directory
When configuring manually, it keeps failing at shared secret key.

This problem is reproducible on multiple accounts and tried three different account and faced the same issue.

hi,
I've got an issue: the autoscaling group does not launch the EC2s Duo Proxies, they are immediately terminated after launch.

STATEREASON Client.InternalError Client.InternalError: instance.log

maybe its because EC2 IAM role "Duo-MFA-for-AWS-Directory-Service-InstanceProfile-xxxxxx" is refrenced but does not exist? I do not find it in IAM.

Thanks for your help.

botocore.vendored and cfnresponse in cloudformation template:

Starting on April 01, 2021, AWS Lambda will no longer support the botocore.requests library in Python runtimes [1][2]. If you are using the cfn-response module for a Lambda-backed custom resource, you must modify your Lambda function resource’s Code or Runtime property and update your stack(s) in order to get the latest version of the cfn-response module which removed its dependency on botocore.requests.

[1] https://aws.amazon.com/blogs/compute/upcoming-changes-to-the-python-sdk-in-aws-lambda/
[2] https://aws.amazon.com/blogs/developer/removing-the-vendored-version-of-requests-from-botocore/
[3] https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-lambda-function-code-cfnresponsemodule.html

There is a character that can be contained in the RADIUS Secret that Directory Service or Lambda does not like (DS RADIUS change simply fails).

I was unable to determine which character it was as I was frantically trying to fix MFA on the Directory although the Secrets cycle caused my Directory Services MFA configuration to fail. Once I cycled the RADIUS secret again, it began working again. I had forced new proxies to spawn but all had the issue.