• Azure AD Application Analytics Solution
• Before using this tool
• Release notes
  • List of checks
• Requirements and operation
  • Operation
  • After running the tool
  • Running in local mode (CSV generation)
• Limitations
• Contribution

Major Refactor of previous solution

• You can read about some of the use cases in the previous solutions documentation solution

Read the MIT license

⚠ Only use this tool if you know what you are doing and have reviewed the code

⚠ Always test the tool first in test environments with non-sensitive data

Beta 
Beta v 0.1.8
- LastSignInTime as per https://learn.microsoft.com/en-us/graph/api/reportroot-list-serviceprincipalsigninactivities?view=graph-rest-beta
Beta v 0.1.7 
 - Azure RBAC assigments directly assigned to SPNs now available 
Beta v 0.1.6
- Added check for SPN assignments via groups
Beta
- Added check for implicit grant
Beta v 0.1.5
- Uses now DefaultAzureCredential to create SAS tokens and upload blobs as per JonneK pull request
Beta v 0.1.0
- Compared to previous version uses now JSON batching and larger resultsize across all queries. 2-3x faster than the previous version
- Release for Azure Security meetup UG

Major performance improvement with JSON batching

1. List app type

2. Collect appOwners from both objects (when both exist) spn and application

3. Collect all credential types from both objects (when both exist) spn and application

4. Review replyUrls for dangling DNS records

• Please note, in case of multitenant app, these values might be outdated (ReplyURL changes are not reflected visibly on the resulting SPN object, but are nonetheless effective)

5. Check if the object has been assigned AAD roles

6. List API permissions in the following format

{ "permissionsReading": [
          "\"AppRole --> api-15764 --> Microsoft Graph - permission: PrivilegedAccess.Read.AzureAD\"",
          "\"AppRole --> api-15764 --> Microsoft Graph - permission: RoleManagement.Read.All\"",
          "\"AppRole --> api-15764 --> Microsoft Graph - permission: PrivilegedAccess.Read.AzureResources\"",
          "\"AppRole --> api-15764 --> Microsoft Graph - permission: PrivilegedAccess.Read.AzureADGroup\"",
          "\"AppRole --> api-15764 --> Office 365 Management APIs - permission: ActivityFeed.Read\"",
          "\"oauth2PermissionGrants --> AllPrincipals --> Microsoft Graph - permission: User.Read\"",
          "\"oauth2PermissionGrants --> AllPrincipals --> Microsoft Graph - permission: Directory.AccessAsUser.All\"",
          "\"oauth2PermissionGrants --> admin santasalo --> Microsoft Graph - permission: User.Read\"",
      ]
    }

7. List Azure RBAC permissions available using by the use of --azRbac option

   example: node main storageAccount --azRbac

   • In the query that is pasted to Log Analytics, you need to expand it as follows by adding azRbac into the project statement

7. Change the project with lastSignIn , and add to the last line extend lastSignInDate = parse_json(lastSignIn).lastSignInActivity.lastSignInDateTime

let home="033794f5-7c9d-4e98-923d-7b49114b7ac3"; 
 //
let admins = (externaldata (id: string, displayName: string, role: string)[@""] with (format="multijson"));
 //
let doNotRemove = (externaldata (test: string)[@""] with (format="multijson"));
// 
// Pre-existing part from query (Dont paste this, just for example here)
//
let servicePrincipalsUP = (externaldata (id: string, deletedDateTime: dynamic, accountEnabled: string, alternativeNames: dynamic, appDisplayName: string, appDescription: dynamic, appId: string, applicationTemplateId: dynamic, appOwnerOrganizationId: string, appRoleAssignmentRequired: string, createdDateTime: string, description: dynamic, disabledByMicrosoftStatus: dynamic, displayName: string, homepage: dynamic, loginUrl: dynamic, logoutUrl: dynamic, notes: dynamic, notificationEmailAddresses: dynamic, preferredSingleSignOnMode: dynamic, preferredTokenSigningKeyThumbprint: dynamic, replyUrls: dynamic, servicePrincipalNames: dynamic, servicePrincipalType: string, signInAudience: string, tags: dynamic, tokenEncryptionKeyId: dynamic, samlSingleSignOnSettings: dynamic, addIns: dynamic, appRoles: dynamic, info: dynamic, keyCredentials: dynamic, oauth2PermissionScopes: dynamic, passwordCredentials: dynamic, resourceSpecificApplicationPermissions: dynamic, verifiedPublisher: dynamic, HasOwner: string, federatedCredentials: dynamic, owners: dynamic, lastSignIn: dynamic, permissions: dynamic, implicitGrant: dynamic, permissionsReading: dynamic, isAdminAADrole: dynamic, appType: string, ApplicationHasPassword: dynamic, ApplicationHasPublicClient: dynamic, allCredentials: dynamic, FullCredentials: dynamic, requiredResourceAccessOnlyPresentOnApps: dynamic, danglingRedirect: dynamic)[@""] with (format="multijson"));
 // // //////// ///////
 // The actual change lastSignIn
servicePrincipalsUP
| project appId, displayName, appType, permissionsReading, allCredentials, owners, isAdminAADrole, danglingRedirect, lastSignIn
| extend includesMultipleCredentialSources = case(tostring(allCredentials) contains "App" and tostring(allCredentials) contains "SPN", true, false)
| extend MultitenantAppWithTenantedCreds = iff(tostring(allCredentials) contains "SPN" and appType !contains "internal" and appType !contains "managed" , true, false )
| extend SharedAppForUserAndAppPermissions = iff(tostring(permissionsReading) contains "AppRole -->" and tostring(permissionsReading) contains "oauth2PermissionGrants -->" , true, false )
| extend lastSignInDate = parse_json(lastSignIn).lastSignInActivity.lastSignInDateTime

Access to Azure Cloud Shell (Bash)

• Permissions to create new storage account or to use existing one.
• Access to Log Analytics workspace
• Azure CLI installed (this get tokens from the underlying Azure CLI installation)
• Storage Blob Contributor Role on the Storage Account

https://learn.microsoft.com/en-us/azure/storage/blobs/assign-azure-role-data-access?tabs=portal

About the generated KQL

• The query is valid for 10 minutes, as SAS tokens are only generated for 10 minutes

Start

git clone https://github.com/jsa2/AADAppAudit

cd AADAppAudit

If you are running the tool in Azure Cloud Shell then all depedencies are already installed

If you want to use with SAS token use this branch instead

# in the folder where the solution was installed 
npm install

node main yourstorageaccountshortname

# paste the code in runtime.kql to the desired log analytics worspace
# "navigate to kql/runtime.kql if code does not open up

Run the pasted query in the workspace

• Remove installation of this service (removes the json files that were stored for the query)
• Delete the resource group (if you provisoned new one) az group delete -n $rg

To only generate CSV, you can run the tool in limited mode, which removes some basic analytic functions that are present in the Log Analytics query.

• Does not require Azure Subscription with storage account (Only AAD access is needed)

• Requires still Azure CLI and Node JS (+14) to be installed on the system this tool is run

  The limited mode returns following data in CSV

  app${delim}appID${delim}aadRole${delim}permissions${delim}danglingRedirect

  

  

  • Changing the delimitter of the CSV you can run the tool as follows node main --delimitter=";"

This tool supports paginated results for the initial batch creation. The later operations which are done by Native MS Graph JSON batching at this point do not look for paginated results. This means that if there is an app, that has more than 999 appRoleAssignments, it will only display the first 999 assignments that are granted for that app (technical limit of these assignments is 1500, so it is possible that some app has been given more than 999 assignments)

• Same applies for app that has more than 999 client secrets.

https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#azure-active-directory-limits

Feel free to open issue or pull requests