README is outdated. Other files may need updates as well, see list below

.gitignore
.travis.yml
API_LIST.yml
DATASET_LIST.yml
ID_MAPPING.csv
README.md
tests.py

https://github.com/NCATS-Tangerine/translator-api-registry/blob/master/pfocr/smartapi.yaml#L758

I plan to swap in pfocrURL to replace figureURL as the ref_url in the registry (once we've updated the source data at BTE).

I would also like to include a new field to continue to pass along the figureURL for potential UI usage, but I'm not sure which field key to use...

Harmonizome has precomputed 139800 gene-disease perturbation associations based on GEO. Would be very useful to add these to the Translator API network.

Example API call to query by gene symbol (retrieving data sets in which it is differentially expressed): http://amp.pharm.mssm.edu/Harmonizome/api/1.0/gene/nanog?showAssociations=true

Example API call to query by data set (retrieving all genes that are differentially expressed): http://amp.pharm.mssm.edu/Harmonizome/api/1.0/gene_set/V/Allen+Brain+Atlas+Adult+Human+Brain+Tissue+Gene+Expression+Profiles

Update the GT_exposures_api to reflect current cmaq-exposures-api. Change name to GT_cmaq_exposures_api

I have APIs with a few datatypes where I'm not clear on the best identifiers to use.

How should I go about finding identifiers for these?

Data Types:

• Text: We have a service that takes two strings, treats them as English MeSH terms and returns articles relating the terms.
• Scores: We have a related service that takes two english terms and returns a score reflecting the "distance" between the words according to a machine learning model. Is there an ontology reflecting concepts like the score?
• Location: We have geo codes expressed as latitude and longitude.
• Time: We have time. For automated service invocation, probably we want time format specific identifiers.

Following up on hackathon discussions about extending the smartAPI spec to capture the 'semantic type' of entities described by data. One thing I would like these extensions/refinements to support is derivation of a simple 'Translator API Catalog' that contains human-readable, summary-level descriptions of API content and accessibility (see #8).

An informal schema for this catalog is here, and describes the elements we would ideally like to extract or derive from the smartAPI files. About half of these elements map directly to existing smartAPI element/fields, so are already achievable (apiName, accessURL, description, license, termsOfService, accessRestrictions).

The semantic typing extensions would ideally support derivation of two additional elements in the catalog schema:

• entityTypes: array of terms describing the entity types that the data returned by the API is about
• entityAssociations: and array of hyphen-separated pairs of entities types describing associations that can be obtained form the API (and optionally a relationship type or entity role)

Other elements in the catalog schema that we would like to derive include:

• dataLevel
• primarySources

But if these are not suited for inclusion in the smartAPI spec, they could be implemented elsewhere - e.g. added under the "translator" element in the API_LIST.yml file, where other Translator-specific metadata currently lives.

@MarkDWilliams will provide details. This is a request from chemistry team

Use an integrated neo4j database to explore how human and machine learning agents might collaborate to extract evidence from knowledge graphs to derive predictions and mechanistic hypotheses.

Tasks

1. Load a neo4j instance with diverse data types from Monarch and SemMed DB databases.
2. Define and optimize cypher 'pathfinding' query templates.
3. Apply templates toward answering selected CQs - start with 'positive control' queries that look for paths through the graph providing evidence supporting a known fact/mechanism (e.g. ALDH2 as a known modifier of FA, or cyclodextrin as a successful re-purposing for Niemann-Pick disease).
4. Manually explore query results by evaluating types of paths returned, defining rules/approaches to identify most meaningful evidence, and refining queries to hone in on these paths in the data.
5. Explore machine learning approaches to automate this process, and derive evidence-based predictions from data in knowledge graphs.
6. Explore approaches/interfaces for human intervention in this process - i.e. how to present underlying rationale for automated predictions in a way that allows human users to evaluate the evidence, refine and extend queries based on this, and inform new experiments and analyses.

Goals

1. Understand data and modeling requirements for this type of approach
2. Inform architectural requirements for BB and reasoner applications - particularly w.r.t automated/machine learning methods that can help weight evidence and make predictions, and interfaces for human intervention in refining and extending ML results.
3. Provide end-to-end examples of what open-ended, ML-guided exploration and discovery in the Translator might look like in practice.

Valued Expertise

1. Monarch/SemMedDB data
2. Cypher query language and graph-based algorithms (e.g. for pathfinding, traversals, edge-weighting)
3. Visualization of graph data and paths
4. Machine learning approaches

Something seems to have gone awry with a recent push:
66676ee
Apparently the GTex API is missing the tags field. It also appears to be a JSON file with a .yaml extension, so I'm not sure quite how that's supposed to be handled.

To avoid this in the future, an owner of the repo could protect the master branch and enforce that tests must be passing before anything can be merged in. @newgene, do you have the power to do that?

Please enter information where you see elipses ... below. The template is a guide, follow it where you can.

Data Source

Optional. Name of the preferred source for this data. E.g. OMIM, DrugBank

• DATABASE: DrugBank

Main Entity Type

What kind of entity are you interested in? Biolink type preferred, e.g. variant, disease, gene

• SUBJECT: drug

Connected Entity Type

What kind of thing should the subject be connected to? E.g. for drug-disease links this would be 'disease'. Biolink type preferred

• OBJECT: disease

• RELATION: treats

Example statement

Enter one or more examples of statements/edges you would expect to retrieve using this source. For example, "aldehydes exacerbates Fanconi anemia"

• EXAMPLE 1: imatinib treats cancer
• EXAMPLE 2: ...

Usage

How will this be used? Give a URL of a workflow if possible

• USAGE: ...

Any other information:

Enter any information you like here

Data Source

Optional. Name of the preferred source for this data. E.g. OMIM, DrugBank

• iPTMnet -- more info at https://research.bioinformatics.udel.edu/iptmnet/

Main Entity Type

What kind of entity are you interested in? Biolink type preferred, e.g. variant, disease, gene

• SUBJECT: Protein

Connected Entity Type

What kind of thing should the subject be connected to? E.g. for drug-disease links this would be 'disease'. Biolink type preferred

• OBJECT: Protein

• RELATION: entity regulated by entity / entity regulates entity

Example statement

Enter one or more examples of statements/edges you would expect to retrieve using this source. For example, "aldehydes exacerbates Fanconi anemia". In future this could be used to drive integration tests

• EXAMPLE 1: CDK2 is phosphorylated by CDK7

From https://research.bioinformatics.udel.edu/iptmnet/api/P24941/substrate, this snippet asserts that CDK7 phosphorylates CDK2 with various bits of provenance

{
  "residue": "T",
  "site": "T165",
  "ptm_type": "Phosphorylation",
  "score": 3,
  "sources": [
    {
      "name": "RLIMS-P",
      "label": "rlimsp",
      "url": "http://research.bioinformatics.udel.edu/rlimsp/"
    },
    {
      "name": "PSP",
      "label": "psp",
      "url": "http://www.phosphosite.org/"
    }
  ],
  "enzymes": [
    {
      "id": "P50613",
      "enz_type": "uniprot_ac",
      "name": "CDK7"
    }
  ],
  "pmids": [
    "18396144"
  ]
}

• EXAMPLE 2: CDK2 phosphorylates MCM4

From https://research.bioinformatics.udel.edu/iptmnet/api/P24941/as-enzyme, this snippet asserts that CDK2 phosphorylates MCM4 with various bits of provenance

{
  "substrate": "P33991",
  "substrate_symbol": "MCM4",
  "site": "S3",
  "score": 0,
  "sources": [
    {
      "name": "HPRD",
      "label": "hprd",
      "url": "http://www.hprd.org/"
    }
  ],
  "pmids": [
    "19651622",
    "16519687",
    "20068231"
  ]
}

Preferred format or ingest method

How would you prefer to get the data? Via API or Data dump that can be ingested into your KG? Would you prefer a smart API registry entry, a neo4j dump, biolink-compliant CSV/RDF/JSON that can be loaded with KGX?

• METHOD: SmartAPI annotation -- note existing Swagger documentation at https://research.bioinformatics.udel.edu/iptmnet/api/doc/

Delete or move some registrations to a "deprecated" branch.

Some registrations/folders are from previous phases of Translator and may not be needed.

registered/previously-updated by Su/Wu lab:

• automat-covid-phenotypes
• broad-pgm
• chembio
• civic/jsonld_context
• clinical_risk_factor
• cohd
• cord
• ebi_ontology_lookup_service_api
• genetics_provider
• harmonize
• hetio
• hmdb
• openfda/jsonld_context
• pharos
• reactome/jsonld_context
• scibite
• scigraph

Duplicate files?
MolePro vs molecular_data_provider

From other teams/previous phases:

• CTD_api
• FooDB_api
• GTEx_api
• GT_cmaq_exposures_api
• GT_endotypes_api
• GT_exposures_api
• _example_api
• api_catalog
• bicluster_api
• depmap
• depmap_bicluster_api
• disease ontology api
• ebi_ontology_mapping_service_api
• ebi_proteins
• ebi_proteins_taxonomy
• ensembl
• gene_knockout_correlation
• greent
• hpotomondo_bicluster_api
• indigo
• rnaseqdb_bicluster_api
• robokop
• robokop_extend
• robokop_messenger
• rtx

http://gnomad-api.broadinstitute.org/

Note that we don't have a strategy for graphql yet...

Please enter information where you see elipses ... below. The template is a guide, follow it where you can.

Data Source

Optional. Name of the preferred source for this data. E.g. OMIM, DrugBank

• DATABASE: pharmkgb

Main Entity Type

What kind of entity are you interested in? Biolink type preferred, e.g. variant, disease, gene

• SUBJECT: drug

Connected Entity Type

What kind of thing should the subject be connected to? E.g. for drug-disease links this would be 'disease'. Biolink type preferred

• OBJECT: drug

• RELATION: interacts_with

Example statement

Enter one or more examples of statements/edges you would expect to retrieve using this source. For example, "aldehydes exacerbates Fanconi anemia". In future this could be used to drive integration tests

• EXAMPLE 1: imatinib interacts_with ??
• EXAMPLE 2: ...

Preferred format or ingest method

How would you prefer to get the data? Via API or Data dump that can be ingested into your KG? Would you prefer a smart API registry entry, a neo4j dump, biolink-compliant CSV/RDF/JSON that can be loaded with KGX?

• METHOD: ...

Usage

How will this be used? Give a URL of a workflow if possible

• USAGE: ...

Any other information:

Enter any information you like here

At present there is no comprehensive and up to date catalog of Translator APIs that provides high-level description of the content and accessibility of data served by each. This has been a barrier for many Translator efforts.

We are asking an API developer or representative to complete a short metadata record describing their API. The schema and templates provided below should allow contributors to enter metadata in 15 minutes or less, and we aim to have complete records for all translator APIs by the end of the Jan 2018 Hackathon.

The metadata in this catalog is intended to complement the more granular metadata collected in the smartAPI registry, to provide summary-level information about the content, accessibility, and utility of each API for technical and non-technical users.

Collection of this metadata will be coordinated with Translator smartAPI registry efforts. Specifically, we will extend the yaml format used the existing API_LIST.yml file with additional fields, as defined in the schema.yaml file.

Instructions for creating a metadata record:

1. Go to the API list at http://bit.ly/apicatalog and add your name to the sign up sheet.
2. Review the schema.yml file, and the example_metadata.yml completed record.
3. Copy text from either the example_metadata.yml record or this template.yml file, and overwrite it in a new file with metadata about your API.
4. Name your metadata file [api name]_metadata.yml and commit it to the api_catalog/records folder in the translator-api-registry repo.

Notes:

1. If a particular field does not apply to your API, enter 'null' as the value.
2. Value set enumerations for fields with controlled entry are provided at the end of the schema.yml file.
3. Feel free to extend the 'Entity Type' value set (ENUM3) directly in the schema file and commit it back with these changes. However, all other value sets are not directly extensible - if you want to alter these value sets please make a ticket proposing the change/extension.

I will hold 'office hour' sessions for at least one hour each day at the hackathon to answer questions and take feedback about the metadata schema and process. Additionally, questions or feedback can be recorded as comments on this ticket.

Once completed, I will merge the content of each API's metadata file into the API_LIST.yml file. From here we can automate derivation of various artifacts (e.g. a spreadsheet view for easier viewing/filtering).

currently have endpoint for going from string to DOID. Extend to use DOID to get parents, children, xrefs. Do initially for diseases, but in theory could be expanded to other types.

(based on discussion with @stuppie and how disease ontology endpoint does this in a somewhat non-intuitive way.)