Code Monkey home page Code Monkey logo

Comments (7)

bashir2 avatar bashir2 commented on July 24, 2024

Thanks @vbothe23 for filing this issue; this is a known problem and the ultimate fix is tracked under #560. That said, extensions are currently half-supported if you are willing to hard-code the extension URL and recompile the code. An example for this is described in PR #562.

from fhir-data-pipes.

LovjeetVyas avatar LovjeetVyas commented on July 24, 2024

@bashir2 can u help me like in where is our data from fhir server is getting transformed in batch
i got some extended keys in my subject like group id, identifier, with nulls

from fhir-data-pipes.

bashir2 avatar bashir2 commented on July 24, 2024

@bashir2 can u help me like in where is our data from fhir server is getting transformed in batch i got some extended keys in my subject like group id, identifier, with nulls

Sorry @LovjeetVyas for the late reply; can you please elaborate more on your issue? Are you saying that you have some FHIR extensions and you are not getting those values in the Parquet files? If that is the case, then this is expected behavior unless if you hardcode the extension URL (as described in #562). The fix for this is in #924 which @chandrashekar-s is working on and is expected to be merged this week.

from fhir-data-pipes.

LovjeetVyas avatar LovjeetVyas commented on July 24, 2024

Hello @chandrashekar-s, @bashir2 I trust this message finds you well. I am reaching out to seek your assistance regarding a matter concerning the compatibility of Google FHIR Data Pipes with the Microsoft FHIR server. Our team has been conducting research and encountering challenges in establishing a connection between the two platforms. I would greatly appreciate your guidance on the requirements for connecting with the Microsoft FHIR server. Additionally, I am attaching a screenshot illustrating the specific issue we are currently facing. Your prompt response and insights would be highly valuable. Thank you, and I look forward to your reply.

BELOW IS THE COMMAND WE ARE USING:

PS D:\ccare\datapipes\fhir-data-pipes> java -jar ./pipelines/batch/target/batch-bundled.jar --fhirServerUrl=https://tenantfhirwrapperdata-tenantfhirservice.fhir.azurehealthcareapis.com --fhirServerOAuthTokenEndpoint=https://login.microsoftonline.com/xysd-sdasdan-xsdasd/oauth2/token --fhirServerOAuthClientId=fhune-s3434n3-sd3dgf45 --fhirServerOAuthClientSecret=uib3b4_34onin23 --resourceList=Patient --batchSize=10 --targetParallelism=2 --fhirSinkPath="" --sinkUserName="" --sinkPassword="" --outputParquetPath=/DATA_WAREHOUSE --jdbcModeEnabled=false --jdbcMaxPoolSize=50 --fhirDatabaseConfigPath=/workspace/utils/hapi-postgres-config.json --jdbcInitialPoolSize=10 --jdbcFetchSize=10000 --jdbcModeHapi=false --runner=DirectRunner --activePeriod="" --since=""

[above is the cli command we are using for connection'

13:07:49.074 [main] INFO com.google.fhir.analytics.FhirEtl com.google.fhir.analytics.FhirEtl.main:389 - Flags: Current Settings:
activePeriod:
appName: FhirEtl
batchSize: 10
fhirDatabaseConfigPath: /workspace/utils/hapi-postgres-config.json
fhirServerOAuthClientId: #############################
fhirServerOAuthClientSecret: #############################
fhirServerOAuthTokenEndpoint: https://login.microsoftonline.com/#######################/oauth2/token
fhirServerUrl: https://ccarefhirwrapperdata-ccarefhirservice.fhir.azurehealthcareapis.com
fhirSinkPath:
jdbcFetchSize: 10000
jdbcInitialPoolSize: 10
jdbcMaxPoolSize: 50
jdbcModeEnabled: false
jdbcModeHapi: false
optionsId: 0
outputParquetPath: /DATA_WAREHOUSE
resourceList: Patient
runner: class org.apache.beam.runners.direct.DirectRunner
since:
sinkPassword:
sinkUserName:
stableUniqueNames: WARNING
targetParallelism: 2

13:07:49.119 [main] INFO ca.uhn.fhir.util.VersionUtil ca.uhn.fhir.util.VersionUtil.initialize:84 - HAPI FHIR version 6.6.2 - Rev 8d55781507
13:07:49.127 [main] INFO ca.uhn.fhir.context.FhirContext ca.uhn.fhir.context.FhirContext.:210 - Creating new FHIR context for FHIR version [R4]
13:07:49.200 [main] INFO c.u.f.c.s.DefaultProfileValidationSupport c.u.f.c.s.DefaultProfileValidationSupportBundleStrategy.loadStructureDefinitions:371 - Loading structure definitions from classpath: /org/hl7/fhir/r4/model/profile/profiles-resources.xml
13:07:49.225 [main] INFO ca.uhn.fhir.util.XmlUtil ca.uhn.fhir.util.jar.DependencyLogImpl.logStaxImplementation:74 - FHIR XML procesing will use StAX implementation 'null' version 'null'
13:07:51.619 [main] INFO c.u.f.c.s.DefaultProfileValidationSupport c.u.f.c.s.DefaultProfileValidationSupportBundleStrategy.loadStructureDefinitions:371 - Loading structure definitions from classpath: /org/hl7/fhir/r4/model/profile/profiles-types.xml
13:07:51.727 [main] INFO c.u.f.c.s.DefaultProfileValidationSupport c.u.f.c.s.DefaultProfileValidationSupportBundleStrategy.loadStructureDefinitions:371 - Loading structure definitions from classpath: /org/hl7/fhir/r4/model/profile/profiles-others.xml
13:07:51.935 [main] INFO c.u.f.c.s.DefaultProfileValidationSupport c.u.f.c.s.DefaultProfileValidationSupportBundleStrategy.loadStructureDefinitions:371 - Loading structure definitions from classpath: /org/hl7/fhir/r4/model/extension/extension-definitions.xml
13:07:52.736 [main] INFO com.google.fhir.analytics.FetchUtil com.google.fhir.analytics.FetchUtil.:82 - Fetching access tokens from https://login.microsoftonline.com/2466764f-5f33-4d63-be86-052226a72c9a/oauth2/token
13:07:53.567 [main] INFO c.g.fhir.analytics.FhirSearchUtil com.google.fhir.analytics.FhirSearchUtil.createSegments:221 - Fetching first batch of Patient
13:07:54.383 [main] ERROR com.google.fhir.analytics.FhirEtl com.google.fhir.analytics.FhirEtl.buildFhirSearchPipeline:141 - Either the date format in the active period is wrong or none of the resources support 'date' featurekey expires_in
Exception in thread "main" java.lang.IllegalArgumentException: key expires_in
at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:900)
at com.google.api.client.json.JsonParser.parse(JsonParser.java:360)
at com.google.api.client.json.JsonParser.parse(JsonParser.java:335)
at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:79)
at com.google.api.client.json.JsonObjectParser.parseAndClose(JsonObjectParser.java:73)
at com.google.api.client.http.HttpResponse.parseAs(HttpResponse.java:460)
at com.google.api.client.auth.oauth2.TokenRequest.execute(TokenRequest.java:324)
at com.google.fhir.analytics.FetchUtil$ClientCredentialsAuthInterceptor.requestAccessToken(FetchUtil.java:196)
at com.google.fhir.analytics.FetchUtil$ClientCredentialsAuthInterceptor.getToken(FetchUtil.java:172)
at com.google.fhir.analytics.FetchUtil$ClientCredentialsAuthInterceptor.interceptRequest(FetchUtil.java:188)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at ca.uhn.fhir.interceptor.executor.BaseInterceptorService$HookInvoker.invoke(BaseInterceptorService.java:517)
at ca.uhn.fhir.interceptor.executor.BaseInterceptorService.doCallHooks(BaseInterceptorService.java:281)
at ca.uhn.fhir.interceptor.executor.BaseInterceptorService.callHooks(BaseInterceptorService.java:269)
at ca.uhn.fhir.interceptor.executor.BaseInterceptorService.callHooks(BaseInterceptorService.java:63)
at ca.uhn.fhir.rest.client.impl.BaseClient.invokeClient(BaseClient.java:319)
at ca.uhn.fhir.rest.client.impl.GenericClient$BaseClientExecutable.invoke(GenericClient.java:538)
at ca.uhn.fhir.rest.client.impl.GenericClient$FetchConformanceInternal.execute(GenericClient.java:830)
at ca.uhn.fhir.rest.client.impl.RestfulClientFactory.validateServerBase(RestfulClientFactory.java:320)
at ca.uhn.fhir.rest.client.impl.RestfulClientFactory.validateServerBaseIfConfiguredToDoSo(RestfulClientFactory.java:283)
at ca.uhn.fhir.rest.client.impl.BaseClient.invokeClient(BaseClient.java:245)
at ca.uhn.fhir.rest.client.impl.GenericClient$BaseClientExecutable.invoke(GenericClient.java:538)
at ca.uhn.fhir.rest.client.impl.GenericClient$SearchInternal.execute(GenericClient.java:1997)
at com.google.fhir.analytics.FhirSearchUtil.createSegments(FhirSearchUtil.java:224)
at com.google.fhir.analytics.FhirEtl.buildFhirSearchPipeline(FhirEtl.java:139)
at com.google.fhir.analytics.FhirEtl.setupAndBuildPipelines(FhirEtl.java:377)
at com.google.fhir.analytics.FhirEtl.main(FhirEtl.java:395)
Caused by: java.lang.IllegalArgumentException: key expires_in, field private java.lang.Long com.google.api.client.auth.oauth2.TokenResponse.expiresInSeconds
at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:900)
at com.google.api.client.json.JsonParser.parse(JsonParser.java:451)
at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:787)
... 29 more
Caused by: java.lang.IllegalArgumentException: number field formatted as a JSON string must use the @JsonString annotation at com.google.common.base.Preconditions.checkArgument(Preconditions.java:143)
at com.google.api.client.util.Preconditions.checkArgument(Preconditions.java:47)
at com.google.api.client.json.JsonParser.parseValue(JsonParser.java:863)
... 31 more

from fhir-data-pipes.

chandrashekar-s avatar chandrashekar-s commented on July 24, 2024

Hi @LovjeetVyas, sorry for the late response. The auth implementation that is followed OAuth2 standards and hence a successful auth response must receive a token in this format, i.e. the expires_in field must be a Long. Even the microsoft implementation follows the same.
Also, can you please create a new issue for this, as the context for this is different.

from fhir-data-pipes.

chandrashekar-s avatar chandrashekar-s commented on July 24, 2024

Support for extensions has been provided with the PR #924, now the fhir profile structure definitions can be configured with the parameter structureDefinitionsDir and the pipelines will use these extensions during conversion to Parquet format. By default, the US Core profile FHIR extensions is supported.

One limitation with the current solution is that, if multiple fhir profiles are configured for the same resource type, then at any given point in time only one profile is applied. A more generic solution is being tracked as part of this issue #980

from fhir-data-pipes.

LovjeetVyas avatar LovjeetVyas commented on July 24, 2024

hey @chandrashekar-s , sorry for this late response I was indulged in some activities and didnt got the time
but so far from our side to authenticate our request we made some changes for that string value that we were getting
and we were getting a url in resource field too that was required for authentication that was not in fhir-datapipes auth files so added that too
and its working fine
another issue that we faced was with pagination
casue ms fhir doesnt support the pagination as hapi fhir and none was specified in the repo and "fhirsearchutil" thus we made some changes into that too
now we are facing a problem with the extension field and currently trying to resolve that too

thankyou for your response and help earlier, if again i got stuck in something will definitely reach out to you guys

from fhir-data-pipes.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.