incatools / relation-graph Goto Github PK
View Code? Open in Web Editor NEWMaterialize OWL existential relations
License: MIT License
Materialize OWL existential relations
License: MIT License
4 columns:
With CURIEs
Why:
It would be helpful (at least for me) if the README
had info about the CLI options (.e.g., non-redundant-output-file
, redundant-output-file
). The relation-graph --help
is a bit terse:
Config
Usage: config [options]
--usage <bool>
Print usage and exit
--help | -h <bool>
Print help message and exit
--ontology-file <string>
--non-redundant-output-file <string>
--redundant-output-file <string>
--mode <output mode>
--property <string*>
--properties-file <string?>
--output-subclasses <boolean value>
--reflexive-subclasses <boolean value>
--equivalence-as-subclass <boolean value>
Can we include the transitive closure of rdfs:subPropertyOf?
there is also an argument for treating domain/range as edges too and including inferred rbox axioms like this
See also:
I understand some are against this, but adding rdfs:labels to output would be helpful. Since OBO ontologies use opaque IRIs, you have to look up the class names if you don't remember the IRI's label ... which often happens for me :)
FWIW, here is a robot command to extract the labels:
robot remove -i input.owl --select annotation-properties --exclude-term rdfs:label \
remove --axioms logical \
-o output-labels.ttl
A sparql query is easy too.
The output would need to be merged with the graph.
cc @balhoff
robot merge -I http://purl.obolibrary.org/obo/upheno.owl -o db/upheno.owl
relation-graph --disable-owl-nothing true \
--ontology-file db/upheno.owl\
--output-file db/upheno-relation-graph.tsv.ttl.tmp \
--equivalence-as-subclass true \
--output-subclasses true \
--reflexive-subclasses true
yields
2022.05.16 16:22:47:341 zio-default-async-1 INFO org.renci.relationgraph.Main.program:57
Running reasoner
2022.05.16 16:28:28:181 zio-default-async-1 INFO org.renci.relationgraph.Main.program:60
Done running reasoner
2022.05.16 16:28:28:257 zio-default-async-1 ERROR org.renci.relationgraph.Main.run:77
Ontology is incoherent; please correct unsatisfiable classes.
From #101 it seems maybe that --disable-owl-nothing
is not sufficient in itself, although this ontology should not have individuals, and the error is about correcting unsatisfiable classes. Perhaps it is the case the RL part does things like check for disjointness violations as rules regardless of owl-nothing?
While we should fix upheno, it would be good to have a mode that is for just calculating the relation graph from nothing more than RBox axioms, subClassOf, someValuesFrom (perhaps classAssertion). While this can be done as a robot step it's convenient to do directly
When I run relation-graph --help
, it produces:
Usage: config [options]
--usage <bool>
Print usage and exit
--help | -h <bool>
Print help message and exit
--ontology-file <string>
--output-file <string>
--mode <output mode>
--property <string*>
--properties-file <string?>
--output-subclasses <boolean value>
--reflexive-subclasses <boolean value>
--equivalence-as-subclass <boolean value>
This is not the same as the README, which includes:
--output-classes <bool>
Output any triples where classes are subjects (default true)
--output-individuals <bool>
Output triples where individuals are subjects, with classes as objects (default false)
--disable-owl-nothing <bool>
Disable inference of unsatisfiable classes by the whelk reasoner (default false)
--prefixes <filename>
Prefix mappings to use for TSV output (YAML dictionary
--obo-prefixes <bool>
Compact OBO-style IRIs regardless of inclusion in prefixes file
--verbose <bool>
Set log level to INFO
cc @balhoff
The core of relation-graph can be extracted and published as a library.
I suspect RG spends a lot of time working through combinations that can be eliminated ahead of time.
E.g. consider a large ontology consisting of protein classes P and go terms T, where the two are linked by P subClassOf involvedIn some T, where T employs a rich RBox such that there are many interesting R some T
s, but there are no named R some P
s. We waste a lot of time if we materialize all R some Ps, we can filter ahead of time and determine there is no named classes inferred to be subclasses.
The following algorithm builds a reduced set of class expressions Xs
to materialize:
Cs = bfsort(Ont.classes)
Rs = bfsort(Ont.properties
Excluded = {}
Xs = []
For r in Rs:
For c in Cs:
if exists r', c' such that r' in {parent(r),r} and c' in {parent(c),c} and Excludes[(r',c')]:
Excluded[(r,c)] = True
Break
Else:
Excluded[(r,c}] = |Reasoner.queryDescendants( r some c )| > 0
Xs.append( r some c )
Intuitively this will give better results in many cases, but may come at a penalty of doing extra computation when Xs approximates the full Cs x Rs cross product. Benchmarks required!
As an aside, I still think the strategy of materializing R-some-Cs is a bit contorted, I suspect you will always get better performance by modeling the RG directly in the ABox and using simple property chains/rules e.g. R o SubClassOf -> R
together with transitivity.. initial experiments with datalog suggest this is the case but not clear if performance is due to the datalog system (Souffle) or the differing strategies (TBox vs ABox)
README has
relation-graph --ontology-file uberon.owl --non-redundant-output-file nonredundant.ttl --redundant-output-file redundant.ttl --mode rdf --property 'http://purl.obolibrary.org/obo/BFO_0000050' --property 'http://purl.obolibrary.org/obo/BFO_0000051' --properties-file more_properties.txt
but the latest has a single output file:
➜ relation-graph git:(master) ./target/universal/stage/bin/relation-graph --help
Config
Usage: config [options]
--usage <bool>
Print usage and exit
--help | -h <bool>
Print help message and exit
--ontology-file <string>
--output-file <string>
--mode <output mode>
--property <string*>
--properties-file <string?>
--output-subclasses <boolean value>
--reflexive-subclasses <boolean value>
--equivalence-as-subclass <boolean value>
it was very useful having both named graphs as separate outputs before - any recommendations for how to get this out with the new CLI?
If I include a non-URI or a URI that does not correspond to an OP in the ontology in my properties file, something silently fails, and the resulting rdf file is empty.
Also: an empty properties file leads to all properties being used. This is potentially useful as a kind of default behavior but maybe surprising.
given:
UBERON: "http://purl.obolibrary.org/obo/UBERON_"
obo: "http://purl.obolibrary.org/obo/"
either emit an error, as the results are non-deterministic, or (preferably) choose the longest namespace, making the shortest CURIE, ie. favor UBERON:nnnn
Capturing in this repo for now. The goal is to have a better categorization of different notions of directness or redundancy of triples in a Relation Graph. Even if these are not implemented by RG immediately, having this vocabulary will help us in discussions of what to include or exclude any given user-facing graph.
The overall goal is to be able to classify triples in a relation graph with categories that help inform users as to whether they are redundant and the nature of the redundancy. E.g. for some applications it is critical to be able to separate "one-hop" edges from edges that are redundant with a multi-hop path (classic Transitive Reduction).
First we define the relation-graph transform (RGT) between an axiom pattern on the left and a triple on the right:
A subClassOf B
where A and B are named classes => A rdfs:subClassOf B
A subClassOf R some B
where A and B are named classes => A R B
A equivalentClass B
where A and B are named classes => A owl:equivalentClass B
(these are a subset of owlstar, with the interpretation triple excluded)
RGT
(O
entailed) = RGRGT
(O
asserted) = RGassertedThe axioms patterns are assumed to match entailed axioms in the input ontology. The original asserted axioms are labeled Oasserted.
A possible extension is to include properties and individuals in the graph, we omit this for now
The set of axioms in O
that do not match either LHS pattern is called O'
(e.g. RBox axioms, logical axioms with nesting or using other constructs)
We use the term "triple" when talking about the output RG triple, and "axiom" when talking about the input OWL ontology axiom. We assume only semantics at the OWL/axiom level and structure at the triple level.
Any triple in the RG must correspond to an axiom (asserted or entailed) in the input ontology according to the two transforms above
Each triple can be categorized according to one or more entailment/redundancy categories below
Additional assumption: the input OWL is coherent. Preprocessing may be applied to reach this state.
A triple t is asserted based on characteristics of it's corresponding axiom a, which matches one of the patterns above
The triple comes from an asserted axioms on the LHS, ie the triple matches an axiom in Oasserted
The triple comes from an entailed axiom on the LHS
note that entailed is not disjoint from asserted. Every asserted triple is necessarily an entailed triple
entailment trivially holds for all edges but we include here for completeness, e.g. we can use it to categorize unseen edges
An example of an entailed triple is A partOf C, given A partOf B, B partOf C, where partOf is transitive in O
(if partOf is not transitive, then the triple is not entailed and hence not in RG)
iff A=B
Note that all reflexive triples SHOULD NOT be asserted for globally reflexive or locally reflexive properties, as the input ontology SHOULD NOT assert these. Presence of these indicates the input MAY come from incorrectly configured robot.
For non-reflexive properties, reflexive triples MAY be asserted. For example, from an asserted axiom neuron subClassOf connected-to some neuron
Either A=Nothing or B=thing or P=topProperty or P=bottomProperty
A default configuration of RG MAY be to always exclude tautologies, and the nodes thing and nothing
Given a triple t0 A R B
, corresponding to axiom a0,
t0 is entailment-direct if there exists no triple t1 corresponding to axiom a1 such thatO-a1 |= a0
this has the effect of excluding both two-hops over transitive predicates, as well as one-hops with more general predicates
A triple A R B is graph-direct if there exists no pair of triples A _ Z, Z _ B
Note this is predicate-blind. It should not be assumed non-onehops encode no useful information. E.g.
A sub C [2-hop]
A partOf B
B sub C
If the 2-hop is exclude it can exclude information that cannot be recapitulated elsehwere, and will impede operations such as inference of a category for a node by subClass traversal
Generalization of above
a triple A R B (corresponding to axiom a0) is a nrpred iff there is no triple
t1 A R' B (corresponding to a1) such that R' != R and
O-a0 |= a0
and NOT: O-(a0 + a1) |= a0
input owl
A sub partOf some B
partOf sub overlaps
reflexive(partOf)
entailed owl
A sub partOf some B
partOf sub overlaps
overlaps sub topProperty
A sub Thing
B sub Thing
A sub A
B sub B
Nothing sub Nothing
Nothing sub A
Nothing sub B
bottomProperty sub partOf
bottomProperty sub overlaps
A sub partOf some A
B sub partOf some B
A sub overlaps some A
B sub overlaps some B
A sub topProperty some A
B sub topProperty some B
A sub overlaps some B
A sub topProperty some B
(we omit irrelevant patterns as the set is of infininite size, eg if we include unions)
triples in RG annotated with categories (omitting 'entailed' categorization)
A partOf B asserted, entailment-direct, graph-onehop, nrpred
A overlaps B, NON-entailment-direct, graph-onehop, redundant-pred
A sub Thing taut, entailment-direct, graph-onehop
B sub Thing taut, non-entailment-direct, non-onehop
Nothing sub A, taut, non-entailment-direct, non-onehop
Nothing sub B, entailment-direct, graph-direct
A partOf A, reflexive
B partOf B, reflexive
A sub A, reflexive
B sub B, reflexive
A overlaps A, reflexive
B overlaps B, reflexive
A topProp A, reflexive, taut
B topProp B, reflexive, taut
Note that if we select ONLY non-taut, entailment-direct from RG we get a single triple A partOf B
This is what a "typical user" might expect from the graph
input owl
A sub partOf some B
B sub partOf some C
transitive(partOf)
triples (excluding tauts and reflexive for brevity)
A partOf B: asserted, entailment-direct, graph-onehop
B partOf C: asserted, entailment-direct, graph-onehop
A partOf C: non-entailment-direct, graph-2hop
contrast with next example
input owl
A sub adjacentTo some B
B sub adjacentTo some C
triples (excluding tauts and reflexive for brevity)
A adjacentTo B: asserted, entailment-direct, graph-onehop
B adjacentTo C: asserted, entailment-direct, graph-onehop
Note this triple is NOT in RG:
A adjacentTo C: NON-ENTAILED, non-entailment-direct, graph-2hop
input owl
A sub negReg some B
B sub negReg some C
negReg sub reg
posReg sub reg
negReg o negReg -> posRef
triples (excluding tauts and reflexive for brevity)
A negReg B: asserted, entailment-direct, graph-onehop
B negReg C: asserted, entailment-direct, graph-onehop
A reg B: non-entailment-direct, graph-onehop
B reg C: non-entailment-direct, graph-onehop
A reg C: non-entailment-direct, graph-2hop, redundant-pred
A posReg C: non-entailment-direct, graph-2hop, nrpred
The intuition here is that even though A posReg C is a 2-hop and can be entailed from existing triple axioms, it is in some sense interesting in that it is a non-redundant pred
wget http://purl.obolibrary.org/obo/cheminf.owl -O owl/cheminf.owl
relation-graph --ontology-file owl/cheminf.owl --output-file inferences/cheminf-inf.ttl
makes a giant stack trace. scroll up and you can see the ontology is incoherent
consider: non-strict option that will drop all constraining axioms if ontology is incoherent. it is awkward to do this with robot as part of a pipeline, and sometimes we just want the graph walking regardless of some bfo nonsense at the top
ttl is more compact but rdf/xml can be used with a wider variety of tools, e.g. rdftab
it's trivial to convert outside RG e.g using riot, but this adds plumbing complexity
For a logical definition like this:
HP:Astrocytosis EquivalentTo RO:has_part some (PATO:increased_rate and (RO:inheres_in_part_of some (GO:cell_growth and (RO:occurs_in some CL:astrocyte))) and (RO:has_modifier some PATO:abnormal)
The term HP:Astrocytosis
will have outgoing edges only where they are inferred to have named class targets, for example RO:has_part PATO:increased_rate
, and any property chain inferences that result from the other nested targets.
Creating nodes for all the sub-expressions would allow more complete query, such as:
?s has_part:/inheres_in_part_of: cell_growth: .
The non-redundant output is not really non-redundant. The code could be simplified if it's just removed. It works better to filter redundancy after the fact, e.g., https://github.com/INCATools/ubergraph/blob/master/prune.dl.
Use case: combine multiple linked ontologies, use RG to determine formal ancestor relationships; avoid accidentally inserting intra-ontology is-as. See: OWL is not modular
This can be done by preprocessing using robot remove with --axioms "equivalent disjoint annotation"
as well as Domain and Range axioms (can't figure out how to do that in robot). It would be convenient to have this as a "graph walker" profile in RG, to avoid preprocessing. This will yield correct n-hop edges while not overwriting vetted assertions from source ontologies.
This issue may subsume:
Reported by @cmungall.
Latest versions of RG use the turtle a
shorthand for rdf:type
This isn't valid n-triples:
https://www.w3.org/TR/n-triples/#grammar-production-IRIREF
Now, strictly speaking this is not a problem. The documentation gives no guarantee about the format, and the output is valid turtle.
However, it looks sufficiently like n-triples that it fools some tools - for example:
➜ semantic-sql git:(adding-sources) ✗ cat foo
<http://www.reactome.org/biopax/81/48887#CellularLocationVocabulary88> a <http://www.biopax.org/release/biopax-level3.owl#CellularLocationVocabulary> .
➜ semantic-sql git:(adding-sources) ✗ riot foo
11:54:09 ERROR riot :: [line: 1, col: 72] Expected IRI: Got: [KEYWORD:a]
For jena, this is obviated by either giving a file suffix OR with an explicit --syntax
However, it still seems a bit of a bait and switch - either use idiomatic nested turtle or use pure n-triples.
It may also be the case that forcing use of a turtle parser would be slower (just checked: it looks like with Jena parsing as turtle vs a using n-triples parser with explicit rdf:types takaes 5% longer, nbd, but other toolchains may be different)
some of this is maybe irrelevant with a TSV format which will allow guaranteed streaming parsing
Example:
@prefix : <http://example.org/> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
:test rdf:type owl:Ontology .
:b rdfs:subClassOf :a .
:c rdfs:subClassOf :b .
:i1 rdf:type :c .
Add an option for triples for i instantiating all 3 classes
Needless to say this should be types at the OWL level - we don't want type owl:Class type owl:NamedIndividual cluttering the output
While RG is awesome, some things I worry about:
I think the use of a README works fine in a local context but things can potentially get confusing when RG triples are remixed back into triples following the standard OWL-RDF serialization. This also has implications for ubergraph - the knowledge that certain queries are transferrable from ubergraph to other triplestores is currently also tacit, rather than having queryable metadata in the triplestore.
I think this could be mitigated by the adoption of some kind of standard. As far as I know the closest we have is https://github.com/cmungall/owlstar
The main anticipated use of owlstar is to annotate individual triples with OWL interpretations, as an alternative to the standard OWL-RDF mapping. Now obviously this would result in massive inflation in size of already massive turtle files (somewhat less if turtlestar is used, but still).
However, there are other ways to achieve the same thing, for example, annotating the graph the triples are contained in; e.g.
<g> os:hasTriplePattern os:SubClassOfSomeValuesFrom , os:TypeSomeValuesFrom, os:SubClassBetweenNamedClasses
this graph-level mechanism would also be a good way of communicating other tacit knowledge about the graph - e.g. that it must include reflexive entailed axioms, that materialization was performed for a certain subset of triples (this bleeds into OMO work, cc @matentzn).
The metadata can be probed by clients to determine if certain kinds of queries are applicable. It can also serve as a mechanism for humans to go and lookup in a standard place what we mean by a triple UBERON:1 BFO:2 UBERON:3
We may still want to switch off graph-annotations by default (it is handy in local contexts to assume that all triples represent edges)
There would still be work to do - e.g algebra of compositions for merging different RGs. But this can come later.
There may be other ways to do this - e.g SHACL shapes?
Install sbt (Scala Build Tool) on your system. For Mac OS X, it is easily done using Homebrew: brew install sbt. sbt requires a working Java installation, but you do not need to otherwise install Scala.
After sbt is installed, run sbt stage to create the executable. The executable is created in your target/universal/stage/bin directory.
After running sbt stage
I get
✗ sbt stage
[info] welcome to sbt 1.6.2 (Homebrew Java 17.0.2)
....
....
[info] Fetched artifacts of
[info] compiling 2 Scala sources to /Users/cjm/repos/relation-graph/core/target/scala-2.13/classes ...
[info] Non-compiled module 'compiler-bridge_2.13' for Scala 2.13.8. Compiling...
[info] Compilation completed in 6.29s.
[info] compiling 4 Scala sources to /Users/cjm/repos/relation-graph/cli/target/scala-2.13/classes ...
[info] Main Scala API documentation to /Users/cjm/repos/relation-graph/cli/target/scala-2.13/api...
[info] Main Scala API documentation successful.
[info] Main Scala API documentation to /Users/cjm/repos/relation-graph/core/target/scala-2.13/api...
[info] Wrote /Users/cjm/repos/relation-graph/core/target/scala-2.13/relation-graph_2.13-2.2.1.pom
[info] Main Scala API documentation successful.
✗ ls -alt target/universal/stage/bin
total 56
drwxr-xr-x 4 cjm staff 128 Feb 11 14:57 .
-rwxr--r-- 1 cjm staff 14579 Feb 11 14:57 relation-graph
-rw-r--r-- 1 cjm staff 10648 Feb 11 14:57 relation-graph.bat
drwxr-xr-x 4 cjm staff 128 Feb 4 16:47 ..
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.