Lost Events about durabletask-mssql HOT 10 OPEN

Hmm...I'm wondering how this might have happened. Just to make sure I understand, your orchestration schedules 60K activities and 14K (23%) of those activities weren't marked as completed in the history?

Were you able to see any warnings or errors in the logs around the time of the problem that might help explain what happened?

from durabletask-mssql.

Sadly not I got it a few Time but didn t found anything related to this :(

from durabletask-mssql.

Today I got the opposite 2 events for same task same orchestration. Deleted second event fixed the orchestration

from durabletask-mssql.

@bhugot by chance are you running this app in Azure? If it's running as an Azure Functions app or an Azure Web App on the Azure App Service platform, we can find the telemetry to help us debug and determine what may have happened.

from durabletask-mssql.

Yes we are on azure.

from durabletask-mssql.

Are you on Azure App Service or Azure Functions? If so, can you let me know the instance ID of one of your orchestrations that had this problem, as well as an approximate time range and the Azure region? I can try to search for the telemetry. Unfortunately, I won't be able to find anything if you're running on VMs or AKS since we don't have telemetry collection for non-App Service platforms.

from durabletask-mssql.

No it s on an azure functions. I should be able to give you those info tomorrow.

from durabletask-mssql.

@cgillum so West Europe, one of this was stuck

from durabletask-mssql.