rorydavidson / snomed-ct-database Goto Github PK

View Code? Open in Web Editor NEW

82.0 82.0 42.0 1.6 MB

Represent SNOMED CT in a relational database

License: MIT License

Shell 10.06% Perl 2.99% Python 76.94% PLpgSQL 0.57% Batchfile 6.60% TSQL 2.73% Awk 0.12%

snomed-ct-database's People

Contributors

Stargazers

Watchers

Forkers

vnisor luismmontielg ebecerraz darkl jianglili007 gnanasundar tuksik madity jrmurray ginsburggregory jeuprefere bartgras alistairewj dbresults snormore liaalabufpb phillette kelsas christillman hmelberg tcj123 ngo010 miguepepper starfishcodemachine taiab alabarga iain-s everdevir kennethmyhra zhaozhw09 bobosui nadjet tersuslabs austinbehan kineticengines davidlrosenblum adib5 gjorgjievc zzti martinklapacz

snomed-ct-database's Issues

Meaning of duplicate entries in curr_simplemaprefset_f? One

Thanks for this project. Tremendously helpful. This is not an issue with the project but more a question about how to interpret data in the SNOMED model. I loaded the S Edition of SNOMED CT(R): March 2017 Release into the schema on PostgreSQL. In the curr_simplemaprefset_f table I have some examples where the same mapping is present with active=0 in one row and active=1 in another row. With different values in the effectivetime column. For example:

id	effectivetime	active	moduleid	refsetid	referencedcomponentid	maptarget
53be7b87-2f7f-5626-a10c-0221bca098f1	20020731	1	900000000000207008	446608001	275990002	C66.9
53be7b87-2f7f-5626-a10c-0221bca098f1	20080131	0	900000000000207008	446608001	275990002	C66.9

Does this mean that this mapping was active beginning "20020731" but then became inactive "20080131"?

Postgresql code is not working

Hello,

Interested in your SQL implementation of SnomedCT I wanted to give it a try with PostgreSQL. This is what i have done (I am a beginner in SQL , so sorry for mistakes or whatever..)

~/download/SNOMED-CT-Database-master%=> creatdb snomedct 
~/download/SNOMED-CT-Database-master%=> psql -d snomedct -f create-database-postgres.sql

So I have modified the load-postgresql.sql according to the location of SnomedCT and changing the delimiter

/* loads the SNOMED CT 'Full' release - replace filenames with relevant locations of base SNOMED CT release files*/
/* Filenames may need to change depending on the release you wish to upload, currently set to January 2015 release */

use snomedct;

COPY curr_concept_f(id, effectivetime, active, moduleid, definitionstatusid) 
FROM '/Users/enricopirani/download/RF2Release/Full/Terminology/sct2_Concept_Full_INT_20150131.txt' 
WITH DELIMITER '~';
CSV HEADER;


COPY curr_description_f(id, effectivetime, active, moduleid, conceptid, languagecode, typeid, term, casesignificanceid)
FROM '/Users/enricopirani/download/RF2Release/Full/Terminology/sct2_Description_Full-en_INT_20150131.txt' 
WITH DELIMITER '~';
CSV HEADER;


COPY curr_textdefinition_f(id, effectivetime, active, moduleid, conceptid, languagecode, typeid, term, casesignificanceid)
FROM '/Users/enricopirani/download/RF2Release/Full/Terminology/sct2_TextDefinition_Full-en_INT_20150131.txt'
WITH DELIMITER '~';
CSV HEADER;


COPY curr_relationship_f(id, effectivetime, active, moduleid, sourceid, destinationid, relationshipgroup, typeid,characteristictypeid, modifierid)
FROM '/Users/enricopirani/download/RF2Release/Full/Terminology/sct2_Relationship_Full_INT_20150131.txt'
WITH DELIMITER '~';
CSV HEADER;


COPY curr_stated_relationship_f(id, effectivetime, active, moduleid, sourceid, destinationid, relationshipgroup, typeid,  characteristictypeid, modifierid)
FROM '/Users/enricopirani/download/RF2Release/Full/Terminology/sct2_StatedRelationship_Full_INT_20150131.txt'  
WITH DELIMITER '~';
CSV HEADER;

COPY curr_langrefset_f(id, effectivetime, active, moduleid, refsetid, referencedcomponentid, acceptabilityid)
FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Language/der2_cRefset_LanguageFull-en_INT_20150131.txt' 
WITH DELIMITER '~';
CSV HEADER;

COPY curr_associationrefset_d(id, effectivetime, active, moduleid, refsetid, referencedcomponentid, targetcomponentid)
FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Content/der2_cRefset_AssociationReferenceFull_INT_20150131.txt'
WITH DELIMITER '~';
CSV HEADER;


COPY curr_attributevaluerefset_f(id, effectivetime, active, moduleid, refsetid, referencedcomponentid, valueid)
FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Content/der2_cRefset_AttributeValueFull_INT_20150131.txt'
WITH DELIMITER '~';
CSV HEADER;

COPY curr_simplemaprefset_f(id, effectivetime, active, moduleid, refsetid,  referencedcomponentid, maptarget);
FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Crossmap/der2_sRefset_SimpleMapFull_INT_20150131.txt'
WITH DELIMITER '~';
CSV HEADER;

COPY curr_simplerefset_f(id, effectivetime, active, moduleid, refsetid, referencedcomponentid)
FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Content/der2_Refset_SimpleFull_INT_20150131.txt' 
WITH DELIMITER '~';
CSV HEADER;

COPY curr_complexmaprefset_f(id, effectivetime, active, moduleid, refsetid, referencedcomponentid, mapGroup, mapPriority, mapRule,  mapAdvice, mapTarget, correlationId)
FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Map/der2_iissscRefset_ComplexMapFull_INT_20150131.txt'
WITH DELIMITER '~';
CSV HEADER;

And loading the sql in snomedct db

~/download/SNOMED-CT-Database-master%=> psql -d snomedct -f load-postgresql.sql

I got this errors

LINE 3: use snomedct;
        ^
psql:load-postgresql.sql:8: ERROR:  relation "curr_concept_f" does not exist
psql:load-postgresql.sql:9: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:14: ERROR:  relation "curr_description_f" does not exist
psql:load-postgresql.sql:15: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:20: ERROR:  relation "curr_textdefinition_f" does not exist
psql:load-postgresql.sql:21: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:26: ERROR:  relation "curr_relationship_f" does not exist
psql:load-postgresql.sql:27: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:32: ERROR:  relation "curr_stated_relationship_f" does not exist
psql:load-postgresql.sql:33: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:37: ERROR:  relation "curr_langrefset_f" does not exist
psql:load-postgresql.sql:38: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:42: ERROR:  relation "curr_associationrefset_d" does not exist
psql:load-postgresql.sql:43: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:48: ERROR:  relation "curr_attributevaluerefset_f" does not exist
psql:load-postgresql.sql:49: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:51: ERROR:  syntax error at or near ";"
LINE 1: ...ive, moduleid, refsetid,  referencedcomponentid, maptarget);
                                                                      ^
psql:load-postgresql.sql:53: ERROR:  syntax error at or near "FROM"
LINE 1: FROM '/Users/enricopirani/download/RF2Release/Full/Refset/Cr...
        ^
psql:load-postgresql.sql:54: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:58: ERROR:  relation "curr_simplerefset_f" does not exist
psql:load-postgresql.sql:59: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;
        ^
psql:load-postgresql.sql:63: ERROR:  relation "curr_complexmaprefset_f" does not exist
psql:load-postgresql.sql:64: ERROR:  syntax error at or near "CSV"
LINE 1: CSV HEADER;

Foreign Keys

The script creates all the tables and populates them all right, but no foreign keys are created. Is this intended? how can I fix this.

Crashes on build

CYPHER_EXECUTION
python /opt/snomed_g/snomed-database-loader/NEO4J//snomed_g_neo4j_tools.py run_cypher build.cypher --verbose --neopw64

*** DB failure: command 1497 [USING PERIODIC COMMIT 200 LOAD CSV with headers from "file:/opt/snomed_g/temp_output/DR_719715003_new.csv" as line with line, line.sctid as source_id, line.destinationId as dest_id, line.rolegroup as rolegroup_id MERGE (rg:RoleGroup { sctid: source_id, rolegroup: rolegroup_id }) WITH line,rg,source_id,dest_id,rolegroup_id MATCH (c:ObjectConcept { sctid: dest_id }) CREATE UNIQUE (rg)-[:RELATIVE_TO_PART_OF { id: line.id, active: line.active, sctid: source_id, typeId: line.typeId, rolegroup: rolegroup_id, effectiveTime: line.effectiveTime, moduleId: line.moduleId, characteristicTypeId: line.characteristicTypeId, modifierId: line.modifierId, history: line.history }]->(c);] : [<class 'py2neo.database.status.ClientError'>,Couldn't load the external resource at: file:/neo4j-community-3.2.6/import/opt/snomed_g/temp_output/DR_719715003_new.csv [Neo.ClientError.Statement.ExternalResourceFailed]]

Import into Neo4j possible artefacts

This may not be a bug but i suspect that it is, when importing Snomed-CT using the Snomed-CT tool some nodes get strange relationships. For example the concept 277644009 is related to 90433002 through an ISA relationship, but the concept 90433002 is also related to 277644009 trough a separate ISA relationship.

[277644009]-ISA->[90433002], [277644009]<-ISA-[90433002]

The SnomedCT version i am using is 20180731T120000Z.

issue about "sequence did not exist, primed"

Hello,

I am trying SNOMED_G tool, encountered the issue as below:

python ./prog/snomed_g_graphdb_build_tools.py db_build --action create --rf2 ./SCT_RF2_201801/ --release_type full --release_type full --neopw64 1234 --output_dir ./output/ --language_code 'en-us' --language_name 'USEnglish'
SNOMED_G bin directory [/Users/wangzhi/Program-test/Neo4j/Snomed_Neo/prog/]
sequence did not exist, primed
JOB_START
FIND_ROLENAMES
FAILED (steps: ['FIND_ROLENAMES'])

I am using:

Python2.7 on a mac;
2.SNOMED CT of version 201801, full

I don't know how to solve this problem. Is there anyone who can help me? :)

Neo4J import 7-2019 release

Hi Rory,

I wanted to give this a go for a demo involving visualization of patient cohorts - but quickly ran into issues.
Running an import stalls on 'FIND_ROLENAMES'. I am running python 3.7.4 and py2neo 3.1.2. The code is executed in a WSL console, and the script does connect to my local Neo4J desktop server.
I have tried python 2.7 & 3.6 without any luck, same results..
I understand this is very old code from a 3rd party - but perhaps you could help me or point me in the right direction!

Best regards, Sander

Console output:
rm /var/tmp/SnomedCT_InternationalRF2_PRODUCTION_20190731T120000Z -R && mkdir /var/tmp/SnomedCT_InternationalRF2_PRODUCTION_20190731T120000Z && python snomed_g_graphdb_build_tools.py db_build --release_type full --mode build --action create --rf2 /c/Users/[]/Documents/Snomed\ release/dir/SnomedCT_InternationalRF2_PRODUCTION_20190731T120000Z/Full/ --release_type full --neopw snomed --output_dir /var/tmp/SnomedCT_InternationalRF2_PRODUCTION_20190731T120000Z
SNOMED_G bin directory [/c/Users/[]/Documents/development/SNOMED-CT-Database/NEO4J/]
sequence did not exist, primed
JOB_START
FIND_ROLENAMES
FAILED (steps: ['FIND_ROLENAMES'])

Build log: build.log

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.