cse-labs / dataops-code-samples Goto Github PK

Deployment instructions to fix:

• missing Microsoft.EventHub from the resource provider list (required for Purview)
• either add installing "azure-cosmos" Python package as a requirement or even better add this to the instructions: "pip3 install --upgrade azure-cosmos": even with python 3.9 and az-cli 2.51 I had this Python error: "no package azure.cosmos"
• wrong reference to repository to clone.
  they refer https://github.com/cse-labs/opsplaybook.git instead of https://github.com/cse-labs/dataops-code-samples.git
• wrong path: it currently is "cd data-mesh-hack\scripts" but should be "cd dataops-code-samples/hackhub/data-mesh-hack/scripts/"
• deployment of Purview does not work:
  • when the option -f true is selected it tries to deploy a free Purview account with error "{"code": "35001", "message": "Validation failed for pview-si6zphb67tygo. A free tier Tenant-level Purview Account already exists for tenant with ID: XXXX, but is not eligible for upgrade. Correlation ID: XXXX"}". I think this has to do with the fact that Purview deployments now default to the free tier and it looks like it might only be available in westcentralus which is different from the region I was trying to deploy to and has capacity issues both for cosmos and for sql server so it cannot be selected. so adding the region (see my other suggestion) would be a workaround
  • when option -f false is selected, the script fails with "ERROR: {"error":{"code":"DeploymentNotFound","message":"Deployment 'azuredeploy-0822-081447' could not be found."}}". In the bicep main you probably need to deal with the case when deployPurview is false as you will not be able to refer to output parameters from such deployment
• also when the main deployment fails (in my case it was because of Purview deployment generating a failure for the above reasons), the script should stop immediately but currently it continues with the deployment, and starts throwing "ERROR: (HttpMethodIsNotSupported) The HTTP method 'POST' is not supported" (it raises it multiple times, so the script continues even if these failures are encountered)

suggestions things we should consider adding/modifying:

• I would suggest adding a random string to the rg name, this way it's easier to debug (you don't have to wait for the other rg to be destroyed before you attempt another deployment)
• the script should have a parameter to set the deployment region, it currently defaults to australiaeast and unless you edit the script it will always deploy there. Ideal solution: we add a script parameter (say -r) to specify the region so you can launch ./deploy.sh -r