I used this repo and created fhir service as app service,

I got FHIR URL as https://name.azurewebsites.net

metadata like https://name.azurewebsites.net/metadata

I used the following repository for iot connector

https://github.com/microsoft/iomt-fhir

But After creating this, I use the following link
https://docs.microsoft.com/en-us/azure/healthcare-apis/fhir/iot-fhir-portal-quickstart

Created IOT Central application and created data export to event hub created from above repository (IOMT Connector)

But I couldnt get the data from
https://name.azurewebsites.net/Observation?code=http://loinc.org|8867-4

We've started experiencing 99%+ failure rate of the MeasurementCollectionToFhir function.
The problem is that the function will time out (after ~20 seconds) on the Observation update in the FHIR server.
The FHIR server is not the problem, in fact we can get updates in FHIR in under 200ms from other services.

The web app machine seems unable to even CURL the endpoint without timeout.

I have currently a Sev A support open with Microsoft Support.

One possible issue could be related to SNAT exhaustion.

https://4lowtherabbit.github.io/blogs/2019/10/SNAT/

Having deployed with the IOMT template, we have no VNET between the IOMT function and the FHIR server.
One of the solutions suggested, besides creating premium VNET, is to improve the app to reuse connections.

My question is whether the IOMT MeasurementCollectionToFhir is reusing connections.
I just deployed the latest code from main, and did not see any improvements

Thanks

Is it ok to rename? Only used internally, seems benign to refactor fixing typo.

Based on the documentation available at Calculatedcontenttemplate, we can choose to use JmesPath expression language instead JSON Path language. However we see this is not working with the data mapper tool.

We feel there are multiple issues around the data mapper tool

1. JmesPath expressions are not working, also tried the custom function fromUnixTimestampMs mentioned in fromUnixTimestampMs
2. Matched Token example provided in the documentation is not working.

4. Data mapper tool is always treating the template type as JSON Path, even if we use Jmes expressions.

{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "PrincipalNotFound",
"message": "Principal 9dcbb7324d6a4a379e48abd91ec982e7 does not exist in the directory 72f988bf-86f1-41af-91ab-2d7cd011db47."
}
]
}

{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"code": "PrincipalNotFound",
"message": "Principal 858935c1d902487793d03f5b593f66a8 does not exist in the directory 72f988bf-86f1-41af-91ab-2d7cd011db47."
},
{
"code": "PrincipalNotFound",
"message": "Principal 858935c1d902487793d03f5b593f66a8 does not exist in the directory 72f988bf-86f1-41af-91ab-2d7cd011db47."
}
]
}

{
"code": "DeploymentFailed",
"message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
"details": [
{
"message": "Encountered an error (ServiceUnavailable) from host runtime."
}
]
}

Successful

Configuration.md documentation is out of date. The links to the source files have changed.

Also, it's not clear what the template is transforming it into. An example of the filled out values on IMeasurement would be helpful.

Thirdly, a method to test and verify transforms based on sample data would be most helpful.

Currently the IoMT Connector publishes normalized data to partitions by device id. If a new patient's data for two different devices arrives at the exact same time the creates on the patient will be issued concurrently potentially leading to duplicate patient records sharing the same identifier.

See #183 and #186 for more details.

As title says.
Add additional property to the bag of supported value types.
Useful to transport attributes from messages to the FHIR observation without having to enumerate all possible values.

Example:
given the message:

{ "Body" : { "patientReliefStarted": "occurred", "reliefType": "forwardRelief" // multiple values possible } }

I want to project all the reliefType values in one string attribute together with the CodeableConcept "patientReliefStarted". Currently it's not possible to achieve.

Tracking issue, I will create a PR for this.

#161 introduced a defect where the updates don't occur when they should. Instead they are mistakenly treated as unchanged and the update is skipped.

The problem is due to mergedObservation and existingObservation being the same object. When existingObservation is passed to MergeObservation internally the same operation reference is modified. The return object is identical, so the subsequent check, if (mergedObservation.IsExactly(existingObservation)) always evaluates to true and the update is skipped.

Hi,

There seems to be some Dependency mismatches when trying to build the data mapper tool using npm.

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: [email protected]
npm ERR! node_modules/eslint
npm ERR! dev eslint@"^7.11.0" from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer eslint@"^5.0.0 || ^6.0.0" from @typescript-eslint/[email protected]
npm ERR! node_modules/@typescript-eslint/parser
npm ERR! dev @typescript-eslint/parser@"^2.5.0" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

I have tried manually installing versions to match the requirements, but am unable to find a set of versions to successfully build with.

I created fhir server using following link
https://github.com/microsoft/fhir-server/blob/main/docs/DefaultDeployment.md

then i got fhir server as app service, data stored in cosmosdb
Then i created iomt fhir using your repo, and put the fhir server url as the above app service link,

After creating all , i just create iot central application in azure with continous patient monitoring template,

Then create export on iot central application, but i cant get the eported data through postman

I have observed a few cases of split observations.

I have mappings that aggregate data on a minute by minute basis:

Instead of having a single observation with the aggregated data for the hour, I sometime observe split observations:

Observation 1:

Observation 2:

As you can observe, the sampledData is accumulated up to a point on the first observation, and continues on the second.

@Team I would like to add a new feature to support the Transformed data to be posted to multiple FHIR servers or at least synchronization service between parent and child FHIR servers, where in the current setup. it supports the data to be posted to only one FHIR server that has been configured while deploying the connector.

pointers for the same is much appreciated.

Ability to specify projection and mapping of arbitrary properties in the source IoT/EventHub message to FHIR Observation extensions.

default-azuredeploy-sandbox.json validation error

getting a PS error on deployment "invalid template" -

    New-AzResourceGroupDeployment : 09:04:04 - Error: Code=InvalidTemplate; Message=Deployment template validation failed: 'The template resource 
  'iomtfhir/Microsoft.Authorization/xxxx-xxx-xxx-xxxx-xxxxx' for type 'Microsoft.Authorization/roleAssignments' at line '331' and column '59' has incorrect segment lengths. A 
  nested resource type must have identical number of segments as its resource name. A root resource type must have segment length one greater than its resource name. Please see 
  https://aka.ms/arm-template/#resources for usage details.'.
  At C:\Users\xxxx\iomt-fhir-master\deploy\scripts\Create-IomtFhirSandboxEnvironment.ps1:101 char:1
  + New-AzResourceGroupDeployment -TemplateFile $sandboxTemplate -Resourc ...
  + 
      + CategoryInfo          : NotSpecified: (:) [New-AzResourceGroupDeployment], Exception
      + FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureResourceGroupDeploymentCmdlet
   
  New-AzResourceGroupDeployment : The deployment validation failed
  At C:\Users\xxxx\iomt-fhir-master\deploy\scripts\Create-IomtFhirSandboxEnvironment.ps1:101 char:1
  + New-AzResourceGroupDeployment -TemplateFile $sandboxTemplate -Resourc ...
  + 
      + CategoryInfo          : CloseError: (:) [New-AzResourceGroupDeployment], InvalidOperationException
      + FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureResourceGroupDeploymentCmdlet

Would be good to add an architecture diagram, showing all Azure services used and how the data flows from end-to-end.

https://github.com/microsoft/iomt-fhir/blob/master/docs/Sandbox.md works for Windows users how about Mac and Linux?

Followed the steps provided for Sandbox deployment.

Running the powershell script ".\Create-IomtFhirSandboxEnvironment.ps1 -EnvironmentName " generates the following error:
New-AzResourceGroupDeployment : 2:32:28 PM - Resource Microsoft.IoTCentral/IoTApps '' failed with message '{
"error": {
"code": "400.030.010.102",
"message": "The geography is not available, or cannot find a matching region to provision the application.
Geography: Central US, manifestId: iotc-patient"
}
}'
At \iomt-fhir\deploy\scripts\Create-IomtFhirSandboxEnvironment.ps1:114 char:1

When running in Create mode with a device ID and no patient ID mapped, MeasurementCollectionToFhir runs successfully without any error, but the data isn't ingested to Azure FHIR.

Similarly, if there is an existing device ID mapped to a patient ID in FHIR, and you ingest data with the same device ID but different patient ID, everything appears to succeed, but the data isn't ingested to FHIR.

In both cases, the MeasurementCollectionToFhir function should indicate an error because it failed to ingest the data to FHIR.

When multiple data inside the same measurment with a ValueQuantity FHIR mapping is proccessed, an exception is thrown.

QuantityFhirValueProcessor is assuming only a value is present. We would like to choose between some strategies (first, last or some aggregation).

We have successfully deployed the connector, but I still haven't figured out the Stream Analytics role. I saw #68, and I was wondering the implications of directly invoking the FHIR conversion function from the normalized data Event Hub, as #68 seems to be doing.

Do you have any plan to merge that pull request? Can we safely get rid of the SA? Does it have anything to do with Output Error Policies for failed jobs? Or is it only meant to efficently support SampledData measurement type?

Sorry for all those questions but I'm trying to clarify my ideas 😅

APIs Involved:
EventHubTrigger, EventHubMeasurementCollector, and IDataNormalizationService

Description:
The NormalizeDeviceData function does not currently handle conditions of unwanted retries. As a result, it may process the same input events multiple times and store duplicate normalized measurements in the output Event Hub. To address this issue, we need to ensure that each input event is processed only once, preventing the insertion of duplicate normalized measurements.

Steps to reproduce:
For each API listed above:

1. Simulate a failure/error exit after that API but before the function returns.
2. Run the function for the first time and let it endure that failure/error exit.
3. Let the function automatically retry with the same input.

Expected behavior:
The NormalizeDeviceData function should process each input event only once and skip any subsequent calls with the same events.

Proposed solution:
Introduce idempotency checks in the NormalizeDeviceData function to ensure that activities are sent only once for each unique message. This can be achieved by:
Though the deviation of the prevalence counter is not a huge problem, such an issue can be resolved by maintaining a LastRequestId field in each table entry and blob entry, which stores the invocation id of the last NormalizeDeviceData function call. The invocation id is constant across Azure Function retries. Before updating a table/blob entry, if the comparison finds item.LastRequestId == FunctionContext.InvocationId, so no more redundant action would be required. Otherwise, if item.LastRequestId != FunctionContext.InvocationId, update the prevalence counter and the LastRequestId field with FunctionContext.InvocationId.

By implementing these changes, the NormalizeDeviceData function will become idempotent and avoid processing the same input events multiple times, ensuring proper usage of the APIs listed above.

Thank you for your contribution to the Github community and I really appreciate your effort in going through this issue.

When using ARM Template Provisioning, Resource Location doesn't show as required field. Not choosing a value for Resource Location it gives an error - can't be deployed to these regions and lists all. Took a while to figure out this is required field.

Followed the steps provided for Sandbox deployment.

Running the powershell script ".\Create-IomtFhirSandboxEnvironment.ps1 -EnvironmentName " generates the following error (most likely occurs only when Azure AD tenant used for app registration is different than tenant tied to your Azure subscription):
New-AzResourceGroupDeployment : 1:00:10 PM - Resource
Microsoft.EventHub/namespaces/eventhubs/providers/roleAssignments
'/normalizeddata/Microsoft.Authorization/3babe616-4482-5441-b294-f0c34074cff7' failed with message '{
"error": {
"code": "PrincipalNotFound",
"message": "Principal 5df336353f664ca5a07aabe547689ad3 does not exist in the directory
72f988bf-86f1-41af-91ab-2d7cd011db47."
}
}'
At \iomt-fhir\deploy\scripts\Create-IomtFhirSandboxEnvironment.ps1:114 char:1

• New-AzResourceGroupDeployment -TemplateFile $sandboxTemplate -Resourc ...

•   + CategoryInfo          : NotSpecified: (:) [New-AzResourceGroupDeployment], Exception
    + FullyQualifiedErrorId : Microsoft.Azure.Commands.ResourceManager.Cmdlets.Implementation.NewAzureResourceGroupDeploymentCmdlet
  

If the observation is still in the cache and deleted from the FHIR server the update flow doesn't work properly and will fail until the observation ages out of the cache.

Switching the image from Windows to Linux and deploying results in the following error in the Azure Portal UI for the IOMT functions

{
    "message": {
        "status": "Failed",
        "error": {
            "code": "ResourceDeploymentFailure",
            "message": "The resource operation completed with terminal provisioning state 'Failed'.",
            "details": [
                {
                    "code": "DeploymentFailed",
                    "message": "At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details.",
                    "details": [
                        {
                            "code": "BadRequest",
                            "message": {
                                "Code": "BadRequest",
                                "Message": "AppSetting with name 'FhirService:Url' is not allowed.",
                                "Target": null,
                                "Details": [
                                    {
                                        "Message": "AppSetting with name 'FhirService:Url' is not allowed."
                                    },
                                    {
                                        "Code": "BadRequest"
                                    },
                                    {
                                        "ErrorEntity": {
                                            "ExtendedCode": "04072",
                                            "MessageTemplate": "AppSetting with name '{0}' is not allowed.",
                                            "Parameters": [
                                                "FhirService:Url"
                                            ],
                                            "Code": "BadRequest",
                                            "Message": "AppSetting with name 'FhirService:Url' is not allowed."
                                        }
                                    }
                                ],
                                "Innererror": null
                            }
                        }
                    ]
                }
            ]
        }
    }
}

This is due to hard-coding ':' when using hierarchical configuration settings which is not valid on Linux where the corresponding format is '__'

It is useful (and in some cases neccessary) to be able to deploy the IOMT connector functions on a linux image.

Requiring a windows image means incurs higher cost (if all other services are deployed on linux images) as this then requires two separate app service plans, as well as two dedicated subnets. etc.

The problem can be easily corrected using GetEnvironmentConfig (I will submit a PR with a potential fix) and the function has been tested to work on Linux in our solution after making this fix. We would however like to PR the fix to this repo as well so that we can continue to apply other updates made to this repo.