Code Monkey home page Code Monkey logo

ontology-development-kit's Introduction

Build the ODK images and run the tests DOI DOI

https://www.wikidata.org/wiki/Q112336713

The Ontology Development Kit (ODK)

Manage your ontology's life cycle with the Ontology Development Kit (ODK)! The ODK is

  • a toolbox of various ontology related tools such as ROBOT, owltools, dosdp-tools and many more, bundled as a docker image
  • a set of executable workflows for managing your ontology's continuous integration, quality control, releases and dynamic imports

For more details, see

Where to get help

Core team

  • @matentzn Nicolas Matentzoglu (Semanticly Ltd)
  • @gouttegd Damien Goutte-Gattat (Flybase)
  • @anitacaron Anita Caron (EMBL-EBI)
  • @balhoff Jim Balhoff (RENCI)
  • @cmungall Chris Mungall (LBNL)
  • @dosumis David Osumi-Sutherland (EMBL-EBI)
  • @ehartley Emily Hartley (Critical Path Institute)
  • @hkir-dev Huseyin Kir (EMBL-EBI)
  • @shawntanzk Shawn Tan (Novo Nordisk)
  • @ubyndr Ismail Ugur Bayindir (EMBL-EBI)

Full list of contributors: https://github.com/INCATools/ontology-development-kit/graphs/contributors

Cite

https://doi.org/10.1093/database/baac087

Outstanding contributions

Outstanding contributors are groups and institutions that have helped with organising the ODK development, providing funding, advice and infrastructure. We are very grateful for all your contributions - the project would not exist without you!

Monarch Initiative

The Monarch Initiative is a consortium of medical, biological and computational experts that provide major ontology services such as the Human Phenotype Ontology, Mondo and an integrative data and analytic platform connecting phenotypes to genotypes across species, bridging basic and applied research with semantics-based analysis.

https://monarchinitiative.org/

European Bioinformatics Institute

The Samples, Phenotypes and Ontologies (SPOT) team, led by Helen Parkinson, is concerned with high throughput mammalian phenotyping, Semantics as a Service and human genetics resources. Members of the SPOT team including David Osumi-Sutherland have made major contributions to ODK, and provided advice, use cases and funding.

https://www.ebi.ac.uk/spot/

University of Florida Biomedical Informatics Program

https://hobi.med.ufl.edu/research-2/biomedical-informatics-3/

Knocean Inc.

Knocean Inc. offers consulting and development services for science informatics, in particular in the area of biomedical ontologies and ontology tooling.

http://knocean.com/

Critical Path Institute

The Critical Path For Alzheimer’s Disease (CPAD) is a public-private partnership aimed at creating new tools and methods that can be applied to increase the efficiency of the development process of new treatments for Alzheimer disease (AD) and related neurodegenerative disorders with impaired cognition and function.

https://c-path.org/

Requirements

Docker

Using the ODK docker image requires Docker Engine version 20.10.8 or greater for v1.3.1.

Tips and Tricks

Customizing your ODK installation

You will likely want to customize the build process, and of course to edit the ontology.

We recommend that you do not edit the main Makefile, but instead the supplemental one (e.g. myont.Makefile) is src/ontology

An example of how you can customise your imports for example is documented here

Migrating an existing ontology repo to the ODK

The ODK is designed for creating a new repo for a new ontology. It can also be used to help figure out how to migrate an existing git repository to the ODK structure. There are different ways to do this.

  • Manually compare your ontology against the template folder and make necessary adjustments
  • Run the seed script as if creating a new repo. Manually compare this with your existing repo and use git mv to rearrange, and adding any missing files by copying them across and doing a git add
  • Create a new repo de novo and abandon your existing one, using, for example, github issue mover to move tickets across.

Obviously the second method is not ideal as you lose your git history. Note even with git mv history tracking becomes harder.

If you have built your ontology using a previous version of ODK, migration of your setup is unfortunately a manual process. In general you do not absolutely need to upgrade your setup, but doing so will bring advantages in terms of aligning with emerging standards ways of doing things. The less customization you do on your repo the easier it should be to migrate.

Consult the CHANGELOG.md file for changes made between releases to assist in upgrading.

More documentation

You will find additional documentation in the src/ontology/README-editors.md file in your repo.

The ODK also comes with built in options to generate your own shiny documentation; see for example the PATO documentation here which is almost entirely autogenerated from the ODK.

Alternative to Docker

You can run the seed script without docker using Python3.6 or higher and Java. See requirements.txt for python requirements.

This is, however, not recommended.

Running OBO dashboard with ODK

Note: this is an highly experimental feature as of ODK version 1.2.24. Note that the display and the scores are under active development and will change considerably in the near future.

Example implementation:

  1. An ODK container wrapper (called odk.sh in the following), similar to the run.sh file in your typical repos src/ontology directory.
  2. A dashboard config YAML file (called dashboard-config.yml in the following)

With both files, you can then create a dashboard using the following command:

sh odk.sh obodash -C dashboard-config.yml

The wrapper (odk.sh) should contain something like the following:

#!/bin/sh
# Wrapper script for ODK docker container.
#
docker run -e ROBOT_JAVA_ARGS='-Xmx4G' -e JAVA_OPTS='-Xmx4G' \
  -v $PWD/dashboard:/tools/OBO-Dashboard/dashboard \
  -v $PWD/dashboard-config.yml:/tools/OBO-Dashboard/dashboard-config.yml \
  -v $PWD/ontologies:/tools/OBO-Dashboard/build/ontologies \
  -v $PWD/sparql:/tools/OBO-Dashboard/sparql \
  -w /work --rm -ti obolibrary/odkfull "$@"

Note that this essentially binds a few local directories to the running ODK container. The directories serve the following purposes:

  1. dashboard: this is where the dashboard is deposited. Look at index.html in your browser.
  2. ontologies: this is where ontologies are downloaded to and synced up
  3. sparql: an optional directory that allows you to add custom checks on top of the usual OBO profile.

This is a minimal example dashboard config for a potential phenotype dashboard:

title: OBO Phenotype Dashboard
description: Quality control for OBO phenotype ontologies. Under construction.
ontologies:
  custom:
    - id: wbphenotype
    - id: dpo
      base_ns:
        - http://purl.obolibrary.org/obo/FBcv
environment:
  ROBOT_JAR: /tools/robot.jar
  ROBOT: robot

The ontologies will, if they exist, be retrieved from their OBO purls and evaluated. There are more options potentially of interest:

title: OBO Phenotype Dashboard
description: Quality control for OBO phenotype ontologies. Under construction.
ontologies:
  custom:
    - id: myont
      mirror_from: https://raw.githubusercontent.com/obophenotype/c-elegans-phenotype-ontology/master/wbphenotype-base.owl
    - id: dpo
      base_ns:
        - http://purl.obolibrary.org/obo/FBcv
prefer_base: True
profile:
  baseprofile: "https://raw.githubusercontent.com/ontodev/robot/master/robot-core/src/main/resources/report_profile.txt"
  custom:
    - "WARN\tfile:./sparql/missing_xrefs.sparql"
report_truncation_limit: 300
redownload_after_hours: 2
environment:
  ROBOT_JAR: /tools/robot.jar
  ROBOT: robot
  • mirror_from allows specifying a download URL other than the default OBO purl
  • base_ns allows specifying the set of namespaces considered to be owned by the ontology (only terms in these namespaces will be evaluated for this ontology. Default is http://purl.obolibrary.org/obo/CAPTIALISEDONTOLOGYID).
  • report_truncation_limit allows truncating long (sometimes HUGE ontology reports) to make them go easier on GITHUB version control.
  • redownload_after_hours: this allows to specify how long to wait before trying to download an ontology (which could be a time consuming process!) again.
  • environment: is currently a necessary parameter but will be made optional in future versions. It allows adding environment variables directly to the config, rather than passing them in as -e parameters to the docker container (both are equivalent though.)
  • profile is an optional parameter that allows specifying your own profile for the quality control (ROBOT) report. By default, this is using the ROBOT report default profile. You can either specify your own profile from scratch, or extend the current default with additional test by using the baseprofile parameter. Find out more about ROBOT profiles here.

A fully working example can be found here.

ontology-development-kit's People

Contributors

allysonlister avatar althonos avatar anitacaron avatar balhoff avatar bjonnh avatar clare72 avatar cmungall avatar cthoyt avatar dosumis avatar dougli1sqrd avatar ehartley avatar gouttegd avatar hkir-dev avatar hrshdhgd avatar jamesaoverton avatar joeflack4 avatar jonmcalder avatar jseager7 avatar kaiiam avatar mariealaporte avatar marijane avatar matentzn avatar nicolevasilevsky avatar nlharris avatar pbuttigieg avatar pcm32 avatar pfabry avatar shawntanzk avatar stroemphi avatar wdduncan avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ontology-development-kit's Issues

Error doing reasoning step

There are no imports - perhaps it expects some?

Preparing initial release, may take a few minutes, or longer if you depend on large ontologies like chebi
EXECUTING: cd src/ontology && make prepare_release && echo SUCCESS || echo FAILURE
robot  reason -i ontology-metadata-edit.owl -r ELK relax reduce -r ELK annotate -V http://purl.obolibrary.org/obo/ontology-metadata/releases/`date +%Y-%m-%d`/ontology-metadata.owl -o ontology-metadata.owl
robot convert -i ontology-metadata.owl -f obo -o ontology-metadata.obo.tmp && mv ontology-metadata.obo.tmp ontology-metadata.obo
cp ontology-metadata.owl ontology-metadata.obo ../.. &&\
	mkdir -p ../../imports &&\
	cp imports/*{owl,obo} ../../imports &&\
	git add ../../imports/*{obo,owl} &&\
	(test -f subsets/*owl && cp subsets/*{owl,obo} ../../subsets && git add ../../subsets/* || echo "no subsets") &&\
	echo "Release files are now in ../.. - now you should commit, push and make a release on github"
cp: imports/*owl: No such file or directory
cp: imports/*obo: No such file or directory
make: *** [prepare_release] Error 1
FAILURE
EXECUTING: git add src/ontology/imports/*{obo,owl}
fatal: pathspec 'src/ontology/imports/*obo' did not match any files
ERROR RUNNING: git add src/ontology/imports/*{obo,owl}
Exiting. Run with '-f' to force execution and ignore errors at ./seed-my-ontology-repo.pl line 203.

Error in running INSTALL.sh - unable to download files

Hi, very new to this, so bear with me if the issue is trivial.

Running the script INSTALL.sh I keep getting errors "ERROR 504: Gateway Time-out" and "ERROR 502: Bad Gateway" when the script run all the wget commands.

Any troubleshooting?

make test should run reasoner only

Currently make test is the same as make all

  • rebuilding imports may be quite expensive on travis
  • odd timing issues can mean that reasoner is not executed (noticed by @marieALaporte )

By default, make test should run reasoner checks over whatever the latest version of the edit file is without attempting to rebuild imports.

foobar.owl and googlecode links survive!

The templating process seems to preserve links to Google Code and foobar.owl in:

products:
- foorbar.owl: https://raw.githubusercontent.com/AgriculturalSemantics/master/agro.owl
- agro.obo: https://raw.githubusercontent.com/AgriculturalSemantics/master/agro.obo

running via Docker

Sometimes I get

make: Warning: File 'Makefile' has modification time 760 s in the future

could be confusing if container and non-container executions are mixed

Extend OSK target repo structure to incorporate design pattern files and related infrastructure

Draft repo structure:

# release files live at the top level
Bar.owl
Bar-simple.owl
.travis.yaml
src/
. patterns/ # DOSDPs used to actively generate ontology live here.
. . Dosdp1.yaml
. . Dosdp2.yaml
. . Dosdp1.tsv
. . Dosdp2.tsv
. . imports/
. . . fu_imports.owl # SLIME generated modules to support
. . pattern_ontology.owl # Ontology of patterns - default fillers. FOR REF ONLY Imports modules
from imports/ in this dir
. . definition_ontology.owl #
. ontology/
. . Makefile # continues to live in ontology directory. Make commands from here drive all ODK processes.
. . README.md # This should be updated with commands to run pattern generation etc.
. . bar-edit.owl # imports definition_ontology.owl + modules in imports dir
. . imports/
. . . fu_imports.owl #

Bundle all tooling using docker

This isn't very scalable

https://github.com/INCATools/ontology-starter-kit/blob/master/.travis.yml#L8-L13

Used all over, with variations:

https://github.com/geneontology/go-ontology/blob/master/.travis.yml#L14-L22

Would it make sense to have a Dockerfile that provided an environment that included

  • owltools
  • robot cc @jamesaoverton
  • dosdp-tools cc @balhoff
  • dosdp python checkers cc @dosumis
  • anything else we add to the chain, e.g. blazegraph loader

This would primarily be for running validating checks on an ontology repo (including reasoner checks, sparql checks, pattern checks) in travis, but may be useful in other contexts

cc @kltm

Some imports owl files not generated without error message

Hi,

I was generating the imports owl files of my ontology by typing "make imports/foo_import.owl" using this Makefile

https://github.com/citmejia/Microbial-Conditions-Ontology/blob/master/src/ontology/Makefile

Some of them were generated. However, imports from EFO, NPO, and SIO were not generated and no error message came out. In addition, when I tried to generate EO owl file the error below came out. What can I do?

Citlallis-MacBook-Pro:ontology citlallimejiaalmonte$ make imports/eo_import.owl
owltools http://purl.obolibrary.org/obo/eo.owl --remove-annotation-assertions -l -s -d --remove-dangling-annotations -o mirror/eo.owl
2018-04-16 11:44:39,372 ERROR (CommandRunner:4646) could not parse:http://purl.obolibrary.org/obo/eo.owl
org.semanticweb.owlapi.io.OWLOntologyCreationIOException: https://raw.githubusercontent.com/Planteome/plant-environment-ontology/master/plant-environment-ontology.obo.owl
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:227)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1250)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1204)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1104)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1060)
at owltools.io.ParserWrapper.parseOWL(ParserWrapper.java:163)
at owltools.io.ParserWrapper.parseOWL(ParserWrapper.java:150)
at owltools.io.ParserWrapper.parse(ParserWrapper.java:132)
at owltools.cli.CommandRunner.runSingleIteration(CommandRunner.java:4634)
at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:76)
at owltools.cli.CommandRunnerBase.run(CommandRunnerBase.java:68)
at owltools.cli.CommandLineInterface.main(CommandLineInterface.java:12)
Caused by: java.io.FileNotFoundException: https://raw.githubusercontent.com/Planteome/plant-environment-ontology/master/plant-environment-ontology.obo.owl
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1926)
at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1921)
at java.security.AccessController.doPrivileged(Native Method)
at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1920)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1490)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStreamFromContentEncoding(AbstractOWLParser.java:170)
at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStream(AbstractOWLParser.java:132)
at org.semanticweb.owlapi.io.AbstractOWLParser.getInputSource(AbstractOWLParser.java:246)
at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser.parse(RDFXMLParser.java:72)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:217)
... 11 more
Caused by: java.io.FileNotFoundException: https://raw.githubusercontent.com/Planteome/plant-environment-ontology/master/plant-environment-ontology.obo.owl
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1872)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at sun.net.www.protocol.http.HttpURLConnection.getHeaderField(HttpURLConnection.java:2982)
at java.net.URLConnection.getContentEncoding(URLConnection.java:523)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getContentEncoding(HttpsURLConnectionImpl.java:410)
at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStream(AbstractOWLParser.java:127)
... 14 more
make: *** [mirror/eo.owl] Error 1

Recommended way to import RO terms?

What would be the recommended way to import RO terms?

I tried to import http://purl.obolibrary.org/obo/RO_0002217

but it seems that none of the ro submodules are imported. Should I go through them one by one and add them manually? ( I would probably not need that many, but that also means that whenever RO updates, I would have to go through all of it again).

Is there a way to import all the imports of ro.owl at once and work from that?

generate non-fake seed terms.txt files

currently these are junk

http://purl.obolibrary.org/obo/XXX_0000001 ## Optional Label
http://purl.obolibrary.org/obo/BFO_0000050 ## part of
http://purl.obolibrary.org/obo/CHEBI_24431 ## chemical entity
http://purl.obolibrary.org/obo/UBERON_0002048 ## lung
http://purl.obolibrary.org/obo/CL_0000540 ## neuron
http://purl.obolibrary.org/obo/GO_0008150 ## biological_process

this causes some versions of robot to trip.

we should have real examples. This can't be static as it depends on user params. osk should do a query for first ~5 in any ontology

Pre-filled import TXTs?

Just curious if the import term TXTs are meant to be pre-populated. Looking at FOODON (cooked up with this kit):

Were these just for seeding?

Allow choice of format for -edit file, including OBO

Given that the OWL-API now consumes OBO with no problems and the some of us still prefer master files to be OBO format for diff/merge/patch, can seed_my_ontology_repo come with an OBO option. Would also be nice to have a choice of other formats, including OFN.

barfing on example

I was trying to set-up a new ontology and got errors pulling-in import ontologies. It looks like OSK pulls in ontologies as '.txt' files but then searches for .owl files to import and barfs on not finding them.
What am I missing?
Relevant snips from command and error message below:

campus-108-115:ontology-starter-kit matthewlange$ ./seed-my-ontology-repo.pl -d po -d ro -d pato -d mmo -u mateolan -t "Food Phenotype Ontology" -c foodphen

WRITING: target/food-phenotype-ontology/src/ontology/imports/po_terms.txt
WRITING: target/food-phenotype-ontology/src/ontology/imports/ro_terms.txt
WRITING: target/food-phenotype-ontology/src/ontology/imports/pato_terms.txt
WRITING: target/food-phenotype-ontology/src/ontology/imports/mmo_terms.txt
WRITING: target/food-phenotype-ontology/src/ontology/reports/README.md


EXECUTING: git init
Initialized empty Git repository in /Users/matthewlange/ontology-starter-kit/target/food-phenotype-ontology/.git/
EXECUTING: git add -A .
EXECUTING: git commit -m 'initial commit of ontology sources of foodphen using ontology-starter-kit' -a
[master (root-commit) 58f0349] initial commit of ontology sources of foodphen using ontology-starter-kit
34 files changed, 735 insertions(+)
create mode 100644 src/ontology/foodphen-edit.owl
create mode 100644 src/ontology/foodphen-idranges.owl
create mode 100644 src/ontology/imports/mmo_terms.txt
create mode 100644 src/ontology/imports/pato_terms.txt
create mode 100644 src/ontology/imports/po_terms.txt
create mode 100644 src/ontology/imports/ro_terms.txt


Preparing initial release, may take a few minutes, or longer if you depend on large ontologies like chebi
EXECUTING: cd src/ontology && make prepare_release && echo SUCCESS || echo FAILURE
robot verify -i foodphen-edit.owl --queries ../sparql/equivalent-classes-violation.sparql ../sparql/trailing-whitespace-violation.sparql ../sparql/owldef-self-reference-violation.sparql ../sparql/xref-syntax-violation.sparql ../sparql/nolabels-violation.sparql -O reports/
org.semanticweb.owlapi.model.UnloadableImportException: Could not load imported ontology: http://purl.obolibrary.org/obo/foodphen/imports/po_import.owl Cause: /Users/matthewlange/ontology-starter-kit/target/food-phenotype-ontology/src/ontology/imports/po_import.owl (No such file or directory)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1690)


Caused by: org.semanticweb.owlapi.io.OWLOntologyCreationIOException: /Users/matthewlange/ontology-starter-kit/target/food-phenotype-ontology/src/ontology/imports/po_import.owl (No such file or directory)
at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:207)


Caused by: java.io.FileNotFoundException: /Users/matthewlange/ontology-starter-kit/target/food-phenotype-ontology/src/ontology/imports/po_import.owl (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)


matthewlange$ ls /Users/matthewlange/ontology-starter-kit/target/food-phenotype-ontology/src/ontology/imports
mmo_terms.txt pato_terms.txt po_terms.txt ro_terms.txt

Test auto-imports

This branch added automation to imports, which initially broke the build. Build then fixed by updating robot to latest snapshot. Now need to test whether automated imports working with this new robot.

OSK barfing on ontology imports

See snippet of error below. INCA_Tools barfs on inclusion of BFO, same happens on PATO, RO, etc.

What to do?

EXECUTING: cd src/ontology && make prepare_release && echo SUCCESS || echo FAILURE robot verify -i pahf-edit.owl --queries ../sparql/equivalent-classes-violation.sparql ../sparql/trailing-whitespace-violation.sparql ../sparql/owldef-self-reference-violation.sparql ../sparql/xref-syntax-violation.sparql ../sparql/nolabels-violation.sparql -O reports/ org.semanticweb.owlapi.model.UnloadableImportException: Could not load imported ontology: Cause: /Users/matthewlange/ontology-starter-kit/target/physical-activity-health-and-fitness-ontology/src/ontology/imports/ro_import.owl (No such file or directory) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1690) at org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxParser.Ontology(OWLFunctionalSyntaxParser.java:231) at org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxParser.parse(OWLFunctionalSyntaxParser.java:171) at org.semanticweb.owlapi.functional.parser.OWLFunctionalSyntaxOWLParser.parse(OWLFunctionalSyntaxOWLParser.java:77) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:197) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1098) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1054) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1004) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntologyFromOntologyDocument(OWLOntologyManagerImpl.java:1015) at org.obolibrary.robot.IOHelper.loadOntology(IOHelper.java:305) at org.obolibrary.robot.IOHelper.loadOntology(IOHelper.java:189) at org.obolibrary.robot.CommandLineHelper.getInputOntology(CommandLineHelper.java:295) at org.obolibrary.robot.CommandLineHelper.updateInputOntology(CommandLineHelper.java:354) at org.obolibrary.robot.CommandLineHelper.updateInputOntology(CommandLineHelper.java:319) at org.obolibrary.robot.VerifyCommand.execute(VerifyCommand.java:132) at org.obolibrary.robot.CommandManager.executeCommand(CommandManager.java:242) at org.obolibrary.robot.CommandManager.execute(CommandManager.java:177) at org.obolibrary.robot.CommandManager.main(CommandManager.java:135) at org.obolibrary.robot.CommandLineInterface.main(CommandLineInterface.java:58) Caused by: org.semanticweb.owlapi.io.OWLOntologyCreationIOException: /Users/matthewlange/ontology-starter-kit/target/physical-activity-health-and-fitness-ontology/src/ontology/imports/ro_import.owl (No such file or directory) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:207) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.actualParse(OWLOntologyManagerImpl.java:1098) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:1054) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadOntology(OWLOntologyManagerImpl.java:957) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.loadImports(OWLOntologyManagerImpl.java:1648) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyManagerImpl.makeLoadImportRequest(OWLOntologyManagerImpl.java:1684) ... 18 more Caused by: java.io.FileNotFoundException: /Users/matthewlange/ontology-starter-kit/target/physical-activity-health-and-fitness-ontology/src/ontology/imports/ro_import.owl (No such file or directory) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(FileInputStream.java:195) at java.io.FileInputStream.(FileInputStream.java:138) at java.io.FileInputStream.(FileInputStream.java:93) at sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90) at sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188) at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStreamFromContentEncoding(AbstractOWLParser.java:165) at org.semanticweb.owlapi.io.AbstractOWLParser.getInputStream(AbstractOWLParser.java:127) at org.semanticweb.owlapi.io.AbstractOWLParser.getInputSource(AbstractOWLParser.java:232) at org.semanticweb.owlapi.rdf.rdfxml.parser.RDFXMLParser.parse(RDFXMLParser.java:72) at uk.ac.manchester.cs.owl.owlapi.OWLOntologyFactoryImpl.loadOWLOntology(OWLOntologyFactoryImpl.java:197) ... 23 more java.lang.IllegalArgumentException: A valid input ontology must be specified at org.obolibrary.robot.CommandLineHelper.updateInputOntology(CommandLineHelper.java:358) at org.obolibrary.robot.CommandLineHelper.updateInputOntology(CommandLineHelper.java:319) at org.obolibrary.robot.VerifyCommand.execute(VerifyCommand.java:132) at org.obolibrary.robot.CommandManager.executeCommand(CommandManager.java:242) at org.obolibrary.robot.CommandManager.execute(CommandManager.java:177) at org.obolibrary.robot.CommandManager.main(CommandManager.java:135) at org.obolibrary.robot.CommandLineInterface.main(CommandLineInterface.java:58) usage: robot [command] [options]  -h,--help                  print usage information  -noprefixes                do not use default prefixes  -p,--prefix           add a prefix 'foo: http://bar'  -P,--prefixes         use prefixes from JSON-LD file  -V,--version               print version information  -v,--verbose               increased logging  -vv,--very-verbose         high logging  -vvv,--very-very-verbose   maximum logging  -x,--xml-entities          use entity substitution with ontology XML                             output commands:  help             print help for command  annotate         annotate ontology  convert          convert ontology  diff             find the differences between two ontologies  export-prefixes  export prefixes to a file  extract          extract terms from an ontology  filter           filter ontology axioms  materialize      materialize ontology  merge            merge ontologies  mirror           mirror ontology imports closure  query            query an ontology  reason           reason ontology  reduce           reduce ontology  relax            relax ontology  repair           repair terms from an ontology  report           report terms from an ontology  template         build an ontology from a template  unmerge          unmerge ontologies  validate-profile validate ontology against an OWL profile  verify           Runs a sparql query on an ontology. Any results of the query are violations, counted, and reported make: *** [sparql_test] Error 1 FAILURE EXECUTING: git add src/ontology/imports/*{obo,owl} fatal: pathspec 'src/ontology/imports/*obo' did not match any files ERROR RUNNING: git add src/ontology/imports/*{obo,owl} Exiting. Run with '-f' to force execution and ignore errors at ./seed-my-ontology-repo.pl line 205.

Update Makefile to support pattern-based ontology development with DOSDP / DOSDP-tools

Spec:

make pattern_imports: Use dictionaries in pattern YAML to generate OWL import seeds. Use these seeds with ROBOT extract commands to generate import files.

Potential problem - How can we reliably specify the ontologies to import from? Should we reply on parsing curies?

make pattern_ontology: DOSDP-tools command TBA + make pattern_imports

make definitions.owl* This needs to wrap generation of all patterns + corresponding TSVs -> a single OWL file.
@balhoff Will dosdp-tools wrap this, or should the Makefile run a set of individual build jobs (one for each pattern) & merge the results using Robot?

make patterns: Wraps all of the above steps; Runs standard travis checks on the edit file + imports.

robot extract should take union of manual tsv list plus all referenced classes

current:

imports/%_import.owl: mirror/%.owl imports/%_terms.txt
	$(ROBOT) extract -i $< -T imports/$*_terms.txt --method BOT -O $(BASE)/$@ -o $@

common practice is to reference a dangling URI in an axiom and derived imports from this.

To compromise we can do the union of referenced + prespecified

GO does this:

seed.txt: enhanced.owl
	$(ROBOT) query -f csv -i $< --select ../sparql/terms.sparql $@

imports/%_terms_combined.txt: imports/%_terms.txt seed.txt
	cat $^ > $@

(the relevance of enhanced.owl is not important here)

the SPARQL is:

SELECT DISTINCT ?term
WHERE {
  { ?s1 ?p1 ?term . }
  UNION
  { ?term ?p2 ?o2 . }
  FILTER(isIRI(?term))
}

robot can't extract the specific modules I need for my ontology

I have already made an issue on the robot repo about this too, but I need advice on how to proceed. It seems that the latest released version of robot is not capable of extracting a specific path between two terms to make imports/foo.owl, which is what I need to replicate what I have in the ontology I'm trying to migrate to this framework. This means that the most similar modules I can extract with robot have more terms than I need. This was because, although I used robot when I developed the ontology outside this framework, I edited manually the MIREOT modules I extracted. Now I realize that this is a bad practice and following the way of the ontology starter kit makes everything smooth and reproducible.

So, my question is, while there is not a possibility to use robot for this purpose, is having bigger imports/foo.owl than those included in the ontology would be an issue for the release process? Currently Is there a possibility of using a non-released version of robot? Or what can I do?

ontodev/robot#282

No README generated

After running seed-my-ontology-repo.pl, it prints a list of next steps, including:

  1. Do not initialize with a README (you already have one)

But there is no README in the template or in the generated project.

efo, npo, sio imports not generated

I have found some inconsistencies when I run make imports/efo_import.owl in my terminal and when it runs on travis. In my terminal no error message arises, but both efo_import.owl and mirror/efo.owl are generated without any classes. Here is a screenshot showing that no error occurrs.

image

When it runs on travis, it raised an error that says that no branch or lower terms files were specified. However, I ran the extract command outside the Makefile with a local download of efo.owl with the same upper and lower terms files and it works.

Here is a screenshot of the travis error.

image

When I toggle block comment one target at a time in the Makefile, this also occurred for NPO and SIO:

image

image

image

image

EDITORS: make: robot: No such file or directory when running example

Reproduce this bug by:

make test1
cd target/my-ontology/src/ontology
make

I get:

make: robot: No such file or directory

Two questions:

  1. Aren't line 86 and line 95 supposed to be $(ROBOT) instead of robot?

  2. Even if I correct the above issue, my terminal still doesn't find the robot command. I have already added it to the PATH and I can use it from the terminal but make still doesn't find it Am I supposed to change line 11 to point the ROBOT variable to where robot is on my system or is there a different way to do this?

imports generation doesn't extract seed from source ontology. It should.

Looks like this is the intent:

# Use ROBOT, driven entirely by terms lists NOT from source ontology
imports/%_import.owl: mirror/%.owl seed.txt
	$(ROBOT) extract -i $< -T imports/$*_terms.txt --method BOT -O $(BASE)/$@ -o $@
.PRECIOUS: imports/%_import.owl

Adding new terms for import to imports/fu_terms.txt is a bit of a faff when editing in Protege. Easier just to add new declaration to source. This is even more of an issue when trying to migrate an existing ontology into this framework or when working with some new application ontology built from various existing ones.

It looks like GO handles this via a SPARQL query to generate a seed file listing classes for import:

seed.txt: enhanced.owl
	$(ROBOT) query -f csv -i $< --select ../sparql/terms.sparql $@

I could add this myself as a pull request, but wondering ability to extract seed list will be built into ROBOT soon.

The -d option does not appear to be optional.

Steps to reproduce:
I ran the following command:
./seed-my-ontology-repo.pl -u openrif -t "Contributor Role Ontology" cro -f

Result:
The script exited with an error. See attached log file. Running the script with the -f flag does not work, either.
seedlog.txt

Expected Result:
Given that the -d flag is described as an option in the help, I expected it to be optional.

"make release" fails

The "make release" command didn't work - at least on my brand of make/bash because the code had $RELEASE instead of $(RELEASE) . I think this is fixed as follows:

release: $(ONT).owl
# $(ONT).obo # disabled for now
        cp $^ $(RELEASEDIR)/.
        mkdir -p $(RELEASEDIR)/imports/ && cp imports/* $(RELEASEDIR)/imports/

Can't find imports folder

Trying to make a new ontology and it looks like at least one folders are missing, and content can't get added.
Error:

cp: cannot stat ‘imports/{owl,obo}’: No such file or directory
make: *** [prepare_release] Error 1
FAILURE
EXECUTING: git add src/ontology/imports/
{obo,owl}
fatal: pathspec 'src/ontology/imports/{obo,owl}' did not match any files
ERROR RUNNING: git add src/ontology/imports/
{obo,owl}
EXECUTING: git add triffo.{obo,owl}
fatal: pathspec 'triffo.{obo,owl}' did not match any files
ERROR RUNNING: git add triffo.{obo,owl}
EXECUTING: git add imports/{obo,owl}
fatal: pathspec 'imports/
{obo,owl}' did not match any files
ERROR RUNNING: git add imports/*{obo,owl}
EXECUTING: git commit -m 'initial release of triffo using ontology-starter-kit' -a

Standardizing ontology .owl metadata

I suggest including certain ontology metadata fields into the ontology starter kit. Our GEEM ontology driven data specification system already takes advantage of the versionIRI field that the Makefile adds in order to version any terms lifted from the ontology, in order to recreate related specifications in the future. As well all the stuff below is displayed with ontology resources when users are browsing them in GEEM - and other systems too I presume - . We added a DC terms date field as well, which is set to mirror the date dumped into the versionIRI, in case the versionIRI has some other scheme to it.

<rdf:RDF 
	…
     xmlns:terms="http://purl.org/dc/terms/"
	…

    <owl:Ontology rdf:about="http://purl.obolibrary.org/obo/genepio.owl">
        <owl:versionIRI rdf:resource="http://purl.obolibrary.org/obo/genepio/releases/2018-02-28/genepio.owl"/>
	…
        <terms:title xml:lang="en">Genomic Epidemiology Ontology</terms:title>
        <terms:description xml:lang="en">The Genomic Epidemiology Ontology aims to provide a comprehensive controlled vocabulary for infectious disease surveillance and outbreak investigations. It is an application ontology that draws on many other ontologies including anatomy, taxonomy, disease, symptoms, environment and food types.</terms:description>
        <oboInOwl:default-namespace rdf:datatype="http://www.w3.org/2001/XMLSchema#string">GENEPIO</oboInOwl:default-namespace>
        <terms:license rdf:resource="http://creativecommons.org/licenses/by/3.0/"/>
    </owl:Ontology>

Right now it seems this kind of stuff would go in https://github.com/INCATools/ontology-starter-kit/blob/master/template/src/ontology/foobar-edit.owl ?

The terms:date gets added too via Makefile:
$(ROBOT) reduce -i $< -r ELK -s true annotate --xml-entities -V $(BASE)/releases/date +%Y-%m-%d/$(ONT).owl --typed-annotation http://purl.org/dc/terms/date date +%Y-%m-%dxsd:date -o "$@"

Bug: mirror files added to repo

If we comment out this git add statement, the mirror files are not added. Seems to be recursive.

from the git documentation

Adds content from all *.txt files under Documentation directory and its subdirectories:

$ git add Documentation/*.txt

Note that the asterisk * is quoted from the shell in this example; this lets the command include the files from subdirectories of Documentation/ directory.

Create a Jupyter environment for running releases

After an ontology developer creates a repo using OSK, they typically do things now via the command line to trigger make builds, usually make all.

There may be some value in allowing a release to be executed via a local Jupyter notebook. This would allow the release manager to interactively explore intermediate reports that are generated as part of the release. These could easily be visualized.

This would work by having a lightweight python library that wraps makefile commands (or maybe switching to snakemake would work here).

An additional possibility is to do this via mybinder.org, obviating the need for the release manager to install robot, owltools, etc. The main challenge here would be authorization to make the push to GH.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.