Comments (6)
Hi Sean,
Thanks for the feedback, and glad it was helpful. Let me respond to your feedback:
- We will look into the confusing message
- The script get_synpuf_files.py used to be in python3 -- in our branch, we converted it to 2.7 for just the reason you mentioned -- consistency. Are you sure you retrieved the unm-improvements branch? The change to 2.7 is documented in the header.
- We didn't know how to get that file either, so we overhauled the program to directly read the OMOP vocabulary files as they come out of the box. Again, are you sure you retrieved the right branch? I can't even find a reference to that file in our branch.
- We did not provide instructions on how to create the OMOP CDM v5 database, as we hadn't got there yet, but I agree it would be helpful to have the full soup-to-nuts instructions.
- Great idea to have a script to run it all.
- I would like to do release the results of running the ETL as a zip file as well. It will be quite large -- any suggestions where?
Thanks!
Christophe
from etl-cms.
Christophe, if you made a version of the dataset using your ETL, the
coordinating center can host it on our amazon instance, and we can expose
it via the OHDSI website. Lee Evans can help with those logistics. Thanks
for your contribution, this is great!
On Thu, Apr 14, 2016 at 7:18 PM, Christophe Lambert <
[email protected]> wrote:
Hi Sean,
Thanks for the feedback, and glad it was helpful. Let me respond to your
feedback:
- We will look into the confusing message
- The script get_synpuf_files.py used to be in python3 -- in our
branch, we converted it to 2.7 for just the reason you mentioned --
consistency. Are you sure you retrieved the unm-improvements branch? The
change to 2.7 is documented in the header.- We didn't know how to get that file either, so we overhauled the
program to directly read the OMOP vocabulary files as they come out of the
box. Again, are you sure you retrieved the right branch? I can't even find
a reference to that file in our branch.- We did not provide instructions on how to create the OMOP CDM v5
database, as we hadn't got there yet, but I agree it would be helpful to
have the full soup-to-nuts instructions.- Great idea to have a script to run it all.
- I would like to do release the results of running the ETL as a zip
file as well. It will be quite large -- any suggestions where?Thanks!
Christophe
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#18 (comment)
from etl-cms.
Hey @ChristopheLambert, well false alarm. I was on 94540d0 from master
, thinking I was on unm-improvements
. No wonder you were so confused, oops! :)
Looks like there's a lead as to where to put the output, excellent. I'll close the issue as most else what I said doesn't seem to apply. Thanks!
from etl-cms.
Patrick, we will be sure to do that.
Sean, glad you reached out anyways. Let us know how it works out!
from etl-cms.
Hi @ChristopheLambert how big is the SYNPUF CDMV5 dataset that you would like to share?
Do you have a preferred way to transfer it? ftp server? I can setup a temporary AWS S3 bucket for you to upload the dataset if needed.
You can send me a direct message on the OHDSI forum, or connect and message me on linkedIn to share the transfer connection details.
Thanks.
from etl-cms.
Hi @leeevans, we are not finished yet, but estimate it will be 110GB uncompressed, and about 18GB compressed. SFTP would be fine.
from etl-cms.
Related Issues (20)
- Column mismatch in create_CDMv5_tables.sql and load_CDMv5_vocabulary.sql HOT 1
- Error on Step 5 of Python ETL process HOT 2
- Person ID as integer, CMS's synthetic data has string IDs HOT 1
- CMS RIF to OMOP ETL? HOT 2
- CMS ACO CCLF --> OMOP V5 HOT 1
- md5 sums don't match files - https://github.com/OHDSI/ETL-CMS/blob/master/python_etl/README.md HOT 4
- Issue while running python code to get the claims data HOT 6
- Memory issue of steps 5 in ETL Script HOT 9
- drug_era, observation_period, cost column mismatch HOT 5
- Update ETL logic to support CDM 6.X HOT 5
- CDMv5.2 is not accessible HOT 2
- KeyError: 'Place of Service' HOT 3
- location_id is null for all care_site records HOT 3
- missing 'DE1_0_2010_Beneficiary_Summary_File_Sample_1.zip' HOT 5
- FTP server down? HOT 1
- Downloading final data HOT 1
- Incorrect URL to one source Synpuf file HOT 5
- Updating to newer CDM versions + upload R script available HOT 1
- key error '40213148'
- Convert CMS SynPUF to OMOP v5.4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from etl-cms.