biothings / biothings_explorer_jsonld Goto Github PK

View Code? Open in Web Editor NEW

4.0 12.0 4.0 113 KB

License: Apache License 2.0

Python 100.00%

ncats-translator

biothings_explorer_jsonld's People

Contributors

Stargazers

Watchers

Forkers

kevinxin90 sulab ismailm quiltomics

biothings_explorer_jsonld's Issues

Remove pandas dependencies from the package

Crawler endpoint returns inconsistent data

This endpoint is returning very odd data:
http://biothings.io/explorer/api/v2/crawler?input_type=mondo&input_value=0018911

{
    "linkedData": {
        "disease": [
            {
                "prefix": "MONDO",
                "object": {
                    "id": "MONDO:0007669",
                    "secondary-id": "HGNC.SYMBOL:renal cysts and diabetes syndrome"
                },
                "predicate": "DiseaseToGeneOrGeneProductAssociation",
                "api": "BioLink API",
                "edge": {
                    "label": "has phenotype",
                    "provided_by": "orphanet.ttl",
                    "id": "RO:0002200"
                }
            },
    ...
    ],
    ...
}

The predicate, edge label, and object ID seem to contradict each other. This is a disease to disease relationship (MONDO:0018911 --> MONDO:0007669), not a disease to phenotype (unless a disease counts as a phenotype?) or disease to gene relationship.

Update README file for the new release

biothings_explorer/setup.py installs package as "biothings_explorer_test"

https://github.com/biothings/biothings_explorer/blob/639280f355dc7a99594ad81a672c0f0d669a5eda/setup.py#L9-L20

This seems like a mistake. Should the package name not be "biothings_explorer"?

API `http://biothings.io/explorer/api/v2/directinput2output` cannot handle "do" disease prefix

E.g. for request url:

http://biothings.io/explorer/api/v2/directinput2output?input_prefix=do&input_value=1498&output_prefix=hp

Response is:

{"message": "Endpoints could be located connecting do to hp. The endpoints are ['https://api.monarchinitiative.org/api/bioentity/disease/{diseaseid}/phenotypes']. However, no output could be found using the endpoint!", "status": 400
}

However, for underlying request endpoint using "ONID" instead of "do":

https://api.monarchinitiative.org/api/bioentity/disease/DOID:1498/phenotypes/

The response is not empty.

Thank you!

List Jupyter Related packages to optional

(see extra_require: https://github.com/biothings/biothings_client.py/blob/master/setup.py)

Weird error when crawling chembl.compound: "The node drugname is not in the digraph"

http://biothings.io/explorer/api/v2/crawler?input_type=chembl.compound&input_value=CHEMBL707

Traceback (most recent call last):
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/networkx/classes/digraph.py", line 779, in successors
    return iter(self._succ[n])
KeyError: 'drugname'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/tornado/web.py", line 1509, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 308, in get
    results = exploreinput(input_type, input_value)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 163, in exploreinput
    endpoints = find_endpoint(_prefix)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 44, in find_endpoint
    return list(bt_explorer.api_map.successors(input_type))
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/networkx/classes/digraph.py", line 781, in successors
    raise NetworkXError("The node %s is not in the digraph." % (n,))
networkx.exception.NetworkXError: The node drugname is not in the digraph.

OSError: Initializing from file failed

http://biothings.io/explorer/api/v2/crawler?input_type=mondo&input_value=0018911

Traceback (most recent call last):
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/tornado/web.py", line 1509, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 308, in get
    results = exploreinput(input_type, input_value)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 160, in exploreinput
    synonyms = IDConverter().find_synonym(input_value, input_type)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/id_converter.py", line 70, in __init__
    self.registry = RegistryParser(readmethod='filepath', initialize=True)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/api_registry_parser.py", line 47, in __init__
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/api_registry_parser.py", line 65, in read_id_mapping_file
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/utils.py", line 39, in readFile
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/pandas/io/parsers.py", line 705, in parser_f
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/pandas/io/parsers.py", line 445, in _read
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/pandas/io/parsers.py", line 814, in __init__
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/pandas/io/parsers.py", line 1045, in _make_engine
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/pandas/io/parsers.py", line 1684, in __init__
  File "pandas/_libs/parsers.pyx", line 391, in pandas._libs.parsers.TextReader.__cinit__
  File "pandas/_libs/parsers.pyx", line 712, in pandas._libs.parsers.TextReader._setup_parser_source
OSError: Initializing from file failed

Crawler exception "TypeError: string indices must be integers"

This endpoint is throwing an exception:
http://biothings.io/explorer/api/v2/crawler?input_type=kegg.pathway&input_value=hsa04950

Traceback (most recent call last):
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/tornado/web.py", line 1509, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 302, in get
    results = exploreinput(input_type, input_value)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 211, in exploreinput
    _output = jh.fetch_properties_by_association_in_nquads(nquads, association_list)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/jsonld_processor.py", line 279, in fetch_properties_by_association_in_nquads
    object_values = self.fetch_object_value_by_predicate_value_in_nquads(nquads, _association)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/jsonld_processor.py", line 247, in fetch_object_value_by_predicate_value_in_nquads
    nquads = nquads['@default']
TypeError: string indices must be integers

change "ID type" input to dropdown box

currently an auto-complete -- makes much more sense as a dropdown

create a tutorial

needs to be linked from front page

mydisease.info/v1/disease/ data lacks gene and variant associations

http://mydisease.info/v1/disease/MONDO:0018911 appears to relate MONDO:0018911 to genes and variants, but the biothings explorer client doesn't illuminate this.

>>> import biothings_explorer_test as bt
>>> locator = bt.APILocator()
>>> [a for a in s if a['endpoint'].startswith('http://mydisease.info/v1/disease/')]
[
{'endpoint': 'http://mydisease.info/v1/disease/{mondoid}', 'predicate': 'EquivalentAssociation', 'subject': {'prefix': 'mondo', 'semantic_type': 'disease'}, 'object': {'prefix': 'omim.disease', 'semantic_type': 'disease'}},
{'endpoint': 'http://mydisease.info/v1/disease/{mondoid}', 'predicate': 'EquivalentAssociation', 'subject': {'prefix': 'mondo', 'semantic_type': 'disease'}, 'object': {'prefix': 'umls.disease', 'semantic_type': 'disease'}},
{'endpoint': 'http://mydisease.info/v1/disease/{mondoid}', 'predicate': 'EquivalentAssociation', 'subject': {'prefix': 'mondo', 'semantic_type': 'disease'}, 'object': {'prefix': 'doid', 'semantic_type': 'disease'}},
{'endpoint': 'http://mydisease.info/v1/disease/{mondoid}', 'predicate': 'HasDescriptionAssociation', 'subject': {'prefix': 'mondo', 'semantic_type': 'disease'}, 'object': {'prefix': 'diseaseDescription', 'semantic_type': 'disease'}},
{'endpoint': 'http://mydisease.info/v1/disease/{mondoid}', 'predicate': 'EquivalentAssociation', 'subject': {'prefix': 'mondo', 'semantic_type': 'disease'}, 'object': {'prefix': 'diseasename', 'semantic_type': 'disease'}}
]

Curies with multiple prefixes

This endpoint returns some odd data entries: http://biothings.io/explorer/api/v2/crawler?input_type=mondo&input_value=0018911

These identifiers (all belonging to EquivalantAssociations) look like errors:

"MESH.DISEASE:ICD10CM:E11.8",
"MESH.DISEASE:KEGG:04950",
"MESH.DISEASE:OMIM:606391",
"MESH.DISEASE:ORDO:552"

If there's a reason for having multiple prefixes (I have also seen duplicated prefixes, e.g. HP:HP:0000089, which I assumed to be a mistake) then please explain.

Also, when I try querying on that first identifier I get an error:

http://biothings.io/explorer/api/v2/crawler?input_type=mesh.disease&input_value=ICD10CM:E11.8

Traceback (most recent call last):
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/tornado/web.py", line 1509, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 308, in get
    results = exploreinput(input_type, input_value)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 160, in exploreinput
    synonyms = IDConverter().find_synonym(input_value, input_type)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/id_converter.py", line 89, in find_synonym
    return self.find_disease_synonym(input_value, input_type)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/id_converter.py", line 176, in find_disease_synonym
    params = self.mydisease_params_template.replace('{input_value}', str(input_value)).replace('{input_type}', MYDISEASE_URI2SCOPE[input_type])
KeyError: 'mesh.disease'

But mesh.disease should be a valid prefix to use.

Keyword searches?

It looks like there are a number of bioentities designated for names:
http://biothings.io/explorer/api/v2/metadata/bioentities

But if I try this: http://biothings.io/explorer/api/v2/crawler?input_type=disease-name&input_value=diabetes
I get an exception:

Traceback (most recent call last):
  File "/home/ubuntu/opt/devpy3/lib/python3.5/site-packages/tornado/web.py", line 1509, in _execute
    result = method(*self.path_args, **self.path_kwargs)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 308, in get
    results = exploreinput(input_type, input_value)
  File "/home/ubuntu/opt/biothings_explorer_web/src/handlers/entitycrawler.py", line 160, in exploreinput
    synonyms = IDConverter().find_synonym(input_value, input_type)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/id_converter.py", line 89, in find_synonym
    return self.find_disease_synonym(input_value, input_type)
  File "/home/ubuntu/opt/biothings_explorer_web/src/biothings_explorer/id_converter.py", line 176, in find_disease_synonym
    params = self.mydisease_params_template.replace('{input_value}', str(input_value)).replace('{input_type}', MYDISEASE_URI2SCOPE[input_type])
KeyError: 'disease-name'

I haven't tried all the name and description bioentity types, but of all the ones I tried this error has been thrown.

The knowledge map has this entry:

{
   "subject":{
      "semantic_type":"disease",
      "prefix":"disease-name"
   },
   "endpoint":"https://pharos.nih.gov/idg/api/v1/diseases/search",
   "object":{
      "semantic_type":"disease",
      "prefix":"pharos.disease"
   },
   "predicate":"EquivalentAssociation"
}

When going to that endpoint I do get data back: https://pharos.nih.gov/idg/api/v1/diseases/search?q=diabetes

Make asynchronous HTTP Requests for all API calls

Currently, the speed of the package when making extensive API calls are very slow. Asynchronous HTTP requests should be implemented to boost the speed.
https://pypi.python.org/pypi/grequests