tayganr / purviewdemo Goto Github PK

The readme file mentions that there should be a pre-populated glossary. I tried this 2 times but the glossary was not deployed.

Deployment is taking 30+ minutes and the access token expires before the deployment is completed.

Current State

• The scripts currently do not check if the required Resource Providers (e.g. Microsoft.Purview, Microsoft.EventHub, Microsoft.Storage, etc) are registered.

Limitation

• When the template is run in an environment with missing Resource Providers, the deployment will fail.
• Users currently have to manually register Resource Providers.

Request

• Add logic within the preDeploymentScript to check/register required Resource Providers automatically.

Current State

• The use of user-assigned managed identities is typically optional for deployment scripts included within ARM templates but mandatory when you need to perform any Azure-specific actions.
• This is currently needed in postDeploymentScript.ps1 (e.g. populate Azure Storage account with sample data Get-AzStorageAccount -ResourceGroupName $resource_group -Name $storage_account_name), hence why a user-assigned managed identity is created as part of the ARM template and passed through.

Limitation

• The script does not currently clean-up/remove this user-assigned managed identity and is left in the resource group.

Request

• Update the script to clean-up/remove the user-assigned managed identity resource automatically once no longer needed.

Current State

• The list of locations is currently static and based off the current list of regions that Azure Purview is available.

Limitation

• Over time, this list will become stale and outdated when compared with the growing list of regions Azure Purview can be deployed to.

Request

• Update the location list so that it dynamically retrieves an up to date list directly from the Microsoft.Purview resource provider.

Example code (needs to be implemented/tested):

$locations = ((Get-AzResourceProvider -ProviderNamespace Microsoft.Purview).ResourceTypes | Where-Object ResourceTypeName -eq accounts).Locations
$location = Get-Random -InputObject $locations

Tried to deploy this template using Azure DevOps with the Service Connection/Service Principal having Contributor Permissions on Subscription level. However, this fails with not having authorized permissions. Can you confirm the deployment requires Owner permissions?

Current State

• The service principal which is created at the beginning of the template remains post deployment.

Limitation

• User needs to manually clean-up/remove service principal (Azure Portal > Azure Active Directory > App Registrations).
• This was previously included in the tail end of the preDeploymentScript but testing seemed to indicate that the service principal was being removed before the data plane operations had complete, causing the overall deployment to fail.

Request

• Update the script to remove the service principal once the data plane operations are complete.

Need to catch the error if a particular location fails for Purview deployment, try another location.

Or could give the user the option to pass in parameter to determine which region to deploy to.

Originally posted by @JWStarkie in #10 (comment)

It looks like the change to Azure AD Graph is here now. Parameter Password has been removed thus failing on line 160
$clientSecret = $sp.secret | ConvertFrom-SecureString -AsPlainText

- Output type has been changed from Microsoft.Azure.Commands.ActiveDirectory.PSADApplication to  #Microsoft.Azure.PowerShell.Cmdlets.Resources.MSGraph.Models.ApiV10.IMicrosoftGraphApplication

- Parameter Password has been removed, customized password is not supported anymore, server assigns secret text when creation`
#

https://azure.microsoft.com/en-us/updates/update-your-apps-to-use-microsoft-graph-before-30-june-2022/
https://docs.microsoft.com/en-us/powershell/azure/azps-msgraph-migration-changes?view=azps-7.0.0

Current state

• until deployment succeed in 1 out of 2, the running animation continues.

Limitation

• if the deployment stops in Azure Portal for whatever reason, the Powershell deployment will forever continue until 'Succeeded' is achieved.

Request

• add in some validation to check if deployment status is cancelled, failed etc. Not sure which would be best to achieve this efficiently (i.e. if/else or switch)

Current State

• Once deployment scripts have completed, the deployment script resource is left behind in the RG.

Limitations

• None present.

Request

• Include script in PostDeployment.ps1 to delete script resource and clean up.

Current state

• resource group and resources deployed to the default AzureContext

Limitations

• could lead to deployment in the wrong subscription

Request
We could either

1. have some selection process as part of deploy or
2. some prerequisites included in README to ensure user has correct subscription selected

Got the below error when deploying. What step(s) is missing?

Current State

• The preDeploymentScript.ps1 automatically selects a random location from a list.

Limitation

• Using the prescribed method on the README, the user does not have an easy way to feed a location of their choice.

Request

• Expose an optional -location parameter for the preDeploymentScript.ps1.
• Include logic to cater for scenarios where the user provides no location.

Hi

When I use the template, I get the following error in the first step. The template selects ukwest region and doesn't let me edit if when I try to select some other region. So I am unable to fix it myself.

The provided location 'ukwest' is not available for resource type 'Microsoft.Purview/accounts'. List of available regions for the resource type is 'eastus,westeurope,southeastasia,canadacentral,southcentralus,brazilsouth,centralindia,uksouth,francecentral,koreacentral,uaenorth,japaneast,switzerlandnorth,westus,southafricanorth,westus3,australiaeast,northeurope,westcentralus,westus2,eastus2'. (Code: LocationNotAvailableForResourceType)

Current State

• Currently data used is not picked up in classification rules.

Limitations

• None as present.

Request

• Use sample data that can be classified by out of the box Azure Purview classifications for richer experience.
• https://mockaroo.com/ or https://dummyapis.com/ may help.

Hi,

I am finding a lot of errors in the provisioning script.

Regards
Rajaniesh

[INFO] ARM Template deployment 2 of 2.
New-AzResourceGroupDeployment: /home/rajaniesh/preDeploymentScript.ps1:216
Line |
216 | -azureActiveDirectoryObjectID $principalId `
| ~~~~~~~~~~~~
| Cannot bind argument to parameter 'azureActiveDirectoryObjectID' because it is null.

[ERROR] Something went wrong with deployment 2.

Current State

• When PreDeployment.ps1 fails, there is no catch for this.

Limitations

• User left in a running indefinite loop waiting for deployment to succeed, but if it never gets to this stage this will never happen.

Request

• Catch errors from the Cloud Shell to halt process or attempt another route for deployment.

Error in Purview deployment:

Line |
  60 | $job = Invoke-RestMethod @params
     |             ~~~~~~~~~~~~~~~~~~~~~~~~~
     | {"error":{"code":"InvalidTemplateDeployment","message":"The template deployment 'deployment-YJlDg' is not valid according to the validation procedure. The tracking id is
     | 'cf701da1-bebc-49f2-9739-306c0af81782'. See inner errors for details.","details":[{"code":"2005","message":"Tenant 72f988bf-86f1-41af-91ab-2d7cd011db47 with 200 accounts has
     | surpassed its resource quota for westeurope location. Please try creating in other available locations or contact support."}]}}```

Current State

• ADLS Gen 2 created with public access enabled.
  Limitations
• None currently.
  Request
• Deploy ADLS Gen 2 with public access disabled and access from within Azure VNet enabled.

Notes:
https://docs.microsoft.com/en-us/azure/templates/microsoft.datalakestore/accounts?tabs=bicep#createdatalakestoreaccountproperties

Add the following as additional prerequisites steps to the deployment;

Enable the following "Resource Providers" at the subscription level;

• Microsoft.Purview
• Microsoft.ContainerInstance

Test connection w/ Lineage extraction (preview) fails

This is due to the newly introduced Lineage extraction (preview). For now, manually provide the db_owner role in Azure SQL Database for your Azure Purview MSI, and create the master key while we add this as an enhancement.

• @isantillan1 @tayganr @leaubl

Current State

• The scripts currently do not accomodate for use with an existing resource group.

Limitation

• The template, by default, deploys a new resource group for all resources.

Request

• Expose an optional -ResourceGroup parameter for the preDeploymentScript.ps1.
• Add logic within the preDeploymentScript to check if a new resource group is required.
• Include logic to cater for scenarios where the user provides no -ResourceGroup parameter.

Current State

• Subscription policies can prevent the deployment of event hub namespaces and storage accounts.

Limitations

• None present at this time.

Request

• Adjust Purview deployment template to allow for deployment in subscriptions with policies in place.