DLT Pipeline failed during Initialization about dbdemos HOT 10 OPEN

Hi, I did an update this morning fixing an install issue (0.2.8), and when I run it on my workspace it's working well.
What could happen is that your init job fails & the model isn't created.

When you install the demo we create a job for you. Can you open it & check its status ? If it's failing can you share what's failing in details ?

Thanks

from dbdemos.

Hi,

Thanks for your prompt response.
I am using 14 days trial databricks version with premium workspace. I am trying to run DLT pipeline but it fails. Attached is the screenshot & error file for detail explaination.

Also got warning: Magic commands (e.g. %py, %sql and %run) are not supported with the exception of %pip within a Python notebook. Cells containing magic commands are ignored. Unsupported magic commands were found in the following notebooks /Users/[email protected]/lakehouse-iot-platform/01-Data-ingestion/01.1-DLT-Wind-Turbine-SQL:

DatabricksDemos_LoadError.txt

from dbdemos.

Hi @rahulwankhade I can't tell you why your update failed to run, but I can tell you the error is not caused by the magic command warning ("unsupported operation"). I see the same message while mine completes successfully (both the first time and with full-refresh-all). The message is only there because mixed-language notebooks are not currently supported, so the python cell is ignored and the companion source notebook 01.2-DLT-Wind-Turbine-SQL-UDF is python-only and registers the python UDF just fine.

from dbdemos.

Thanks for your response.

Could you please let me know if any additional configurations requires to run DLT pipeline?
If yes then where parameters needs to be change & which places?

from dbdemos.

The error indicates that the model wasn't loaded.
To understand why you need to open the init job. Open the workflow UI, select the job named
dbdemos_lakehouse_iot_turbine_init_<your_name>

Step 3 register the ML model. Something likely failed during your initialization.
Can you share the error ?

Thanks

from dbdemos.

Job run fails 4th Stage (Start_dlt Pipeline).

Error:
DataPlaneException: [DLT ERROR CODE: CLUSTER_LAUNCH_FAILURE.CLOUD_PROVIDER_FAILURE] Failed to launch pipeline cluster 0329-094505-9twihzze with termination code CLOUD_PROVIDER_LAUNCH_FAILURE and termination type CLOUD_FAILURE: Could not launch cluster due to cloud provider failures. azure_error_code: OperationNotAllowed, azure_error_message: Operation could not be completed as it results in exceeding approved Total Regio...

This seems due to I am using Databricks Trial version (14days).

Trial version has limitations to run DLT pipeline?

from dbdemos.

Where are you saying it's Trial I don't see it in the error (your image doesn't load) ?
Do you have the end of the error ? Looks like an Azure quota no ?

from dbdemos.

Please refer attached error file along with Screenshot.

DLT_PipelineFails_description.txt

from dbdemos.

yes so now your model is loaded, if you go back to your DLT pipeline you'll find the full error.
You can click on the last read "start_dlt_pipeline" task, then click on the DLT pipeline link, and then get the message from the DLT run.
It looks like an Azure VM quota

from dbdemos.

@rahulwankhade I had the same error and it was due to vCPU usage going over the Azure subscription quota limit as pointed out in the message above. You can increase the limits by going to Subscription > Usage + Quotas.

from dbdemos.