Comments (2)
chandrashekar-s
commented on July 24, 2024
Hi @Charantl, thanks for filing this issue. Based on the current design of fhir-data-pipes, if a pipeline fails then the next pipeline that runs will start picking up records from the point where the previous pipeline was successfully complete (i.e. from the start time of the previous successful run as that would be the cut off time for that run). Based on this design, I think the scenario of running the pipeline for a specific data range might not be needed as there wont be records left out even if the pipeline fails (as they will be picked up in the next run).
Do you still think that this feature is needed? If yes, can you please specify the use case for the same.
from fhir-data-pipes.
Charantl
commented on July 24, 2024
Yes @chandrashekar-s. If data for a specific date(s) is removed due to human error, we need a feature to reload the data for that particular date (range).
from fhir-data-pipes.
Related Issues (20)
- NPE while trying to find StructureDefinition for a resource extensions that is missing.
- Integrate FHIR Bulk export API with data-pipes
- Add config to change the pipeline controller cron schedule timezone HOT 1
- add config to override the default 100 batch size for server queries HOT 1
- Add last run time and purging schedule to the pipeline contoller UI
- Various documentations
- Feature Request: Pipeline Failure Notifications
- Relook at the `recursiveDepth` parameter
- Fix the number of parquet files generation for Direct Runner
- Create a Util Function for testing Resources in the FHIR store
- Add ability to turn on /off parquet file generation in case of syncying fhir to fhir server
- Support BULK_EXPORT mode of fetching FHIR resources for Incremental Run
- Controller maven module unit tests in `Junit 4` does not report coverage correctly
- Make sure all `DoFn`s are idempotent
- Migrate all unit test cases to Junit 5 platform HOT 5
- Change the default for batchSize to 3000
- FhirEtl misses resources HOT 9
- Support Parquet as an option for output flat views
- Extra Documentation for mapping ViewDefinitions to DB tables
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fhir-data-pipes.