Access to inferred triples about pyshacl HOT 15 CLOSED

@JoelBender
I've started implementing some of the SHACL Advanced Features, specifically I've started on the Triple-Rules feature. I have it mostly working and passing some tests now, but there are still lots of work to go to pass the SHACL-AF test suite.

from pyshacl.

Hi @houzw
When using the ont_graph feature, and shacl-advanced features at the same time, the expanded_graph will always contain the extra triples mixed in from the extra-ontology graph, and the extra inferred triples from SPARQLRule. That is just the way pySHACL works.

In your first example code snippet, you are not enabling Advanced mode. Try this:

v = Validator(data_graph=dg_basin, shacl_graph=rule, options={"inference": "rdfs", "advanced": True} ,ont_graph=ont)
conforms, report_graph, report_text = v.run()

In your second example, you can disassemble the results like this:
conforms, report_graph, report_text = validate(data_graph=dg_basin, shacl_graph=rule, ont_graph=ont, advanced=True, inference='rdfs', debug=False)
But you can't get expanded_graph that way.

from pyshacl.

Hi @JoelBender

pySHACL does do the described inferencing as per section 8.4, but it doesn't do that by default, it only does the inferencing if the feature is enabled.
EDIT: SHACL can do RDFS and OWL-RL inferencing, but not SHACL Rule Inferencing as per section 8.4.

If you're using the commandline tool, it can be enabled by passing -i rdfs or -i owlrl or -i both option to the executable on the commandline. If you're using the python library within an application, you can use validate(..., inference="rdfs") (or "owlrl", or "both") in the validate helper function to enable the feature.

pySHACL uses the OWL-RL library to do the inferencing step. They way OWL-RL works is it simply adds all of the inferred triples into the graph it is working on. This is known as "graph expansion".

Your question however is about getting access to the data graph containing the inferred triples.

If you are using the commandline tool, this is not possible. There is no feature (nor is there one planned) to return the expanded data graph back to the user via the commandline.

If however you are using the python library within your application, then it is possible to get the expanded data graph, but you can't do it using the validate() helper function. You will instead need to create an instance of the pySHACL Validator class yourself, then access the datagraph on that, like so:

v = Validator(mydata, shacl_graph=myshapes, options={"inference": "rdfs"})
conforms, report_graph, report_text = v.run()
expanded_graph = v.target_graph #<-- This gets the expanded data graph

from pyshacl.

@JoelBender
I've just noticed a bug in pySHACL that means the code above will give you the original data graph, not the expanded graph, that will be fixed in the next release.

I've added an issue tracking that here: #21

from pyshacl.

@JoelBender
That issue is fixed now. To that code above should work to retrieve the expanded graph from the Validator class.

from pyshacl.

Given this code:

data_graph = Graph()
data_graph.parse('bakery.ttl', format='turtle')

shacl_graph = Graph()
shacl_graph.parse('bakeryRules.ttl', format='turtle')

v = Validator(data_graph, shacl_graph=shacl_graph, options={"inference": "rdfs"})
expanded_graph = v.target_graph

The data_graph and the expanded_graph are the same graph, which makes sense assuming that most people want in-place inference. This version:

v = Validator(data_graph, shacl_graph=shacl_graph, ont_graph=Graph(), options={"inference": "rdfs"})
expanded_graph = v.target_graph

The expanded_graph contains the additional content, and the xor of the two gives me the additional triples that were added (the "inference only graph"). However, the library does not infer that AppleTartC is a NonGlutenFreeBakedGood, and I'm not sure if this is a feature of the Jena library that is not supported, or if it's a bug.

from pyshacl.

Ah! I've found that sh:TripleRule is part of the SHACL Advanced Features along with sh:condition, so this magically becomes an enhancement request :-).

from pyshacl.

@JoelBender
I'm having trouble following your examples.
The inferencer does not run until you execute v.run()
When you do:

v = Validator(data_graph, shacl_graph=shacl_graph, options={"inference": "rdfs"})
expanded_graph = v.target_graph

You say that data_graph and expanded_graph is the same, that is expected, because you have not done v.run() yet.

You also have:

v = Validator(data_graph, shacl_graph=shacl_graph, ont_graph=Graph(), options={"inference": "rdfs"})
expanded_graph = v.target_graph

And say that expanded_graph now has the additional content. That is not possible. Two reasons:

1. Adding an empty ont_graph does nothing to change how the inferencing works. The result will be the same as the first snippet.
2. You have not done run() yet so the inferencing has not occurred yet. There is no additional content in the target graph at this point.

Ah! I've found that sh:TripleRule is part of the SHACL Advanced Features along with sh:condition, so this magically becomes an enhancement request :-).

I'm not sure how this issue relates to the sh:TripleRule feature of SHACL. Can you explain more about what you are trying to achieve?

from pyshacl.

Ah, sorry, that was a copy/paste error, a gist of the application is here. Adding the empty ont_graph triggers the data_graph to be "cloned" so I can find just the new triples afterwards, by "additional content" I was referencing the triples that are not in the data graph. The DZone article uses the advanced feature to add the additional triple for each of the tarts and I was expecting it to appear in the target graph.

from pyshacl.

@JoelBender
Ok, it looks like I was getting confused between RDFS/OWL-RL Inferencing and SHACL Rule-inferencing as per that Advanced Features spec.
I've edited my comment up above to clarify that pySHACL does not do SHACL Rule-inferencing as per section 8.4. It can only do the more common/traditional RDFS and OWL-RL based inferencing.
I've actually didn't even know that SHACL Rule-based inferencing exists. I've never seen that advanced-features spec before.

Its looks like the TripleRule feature should be fairly easy to implement, but I'm not sure how useful it will be, or how much use it will get.

from pyshacl.

@JoelBender
I've pushed a new version of PySHACL (v0.10.0 ) that includes some of the features outlined in the SHACL Advanced Features spec.
Specifically, it includes support for sh:TripleRule and sh:SPARQLRule along with, sh:condition.
To use these features, you need to supply --advanced to the commandline tool, or advanced=True to the python lib validate() invocation.

If this feature is still applicable to your use-case, please test it and let me know if it works for you.

from pyshacl.

@JoelBender Closing this issue now, I believe it to be resolved since the addition of SHACL Advanced Features.

from pyshacl.

@ashleysommer

Still cannot get the inferred triples (SPARQLRule).
I have tried the following code

v = Validator(data_graph=dg_basin, shacl_graph=rule, options={"inference": "rdfs"},ont_graph=ont)
conforms, report_graph, report_text = v.run()
expanded_graph = v.target_graph 

but got the content of ont_graph.
If I try:

r = validate(data_graph=dg_basin, shacl_graph=rule, ont_graph=ont, advanced=True, inference='rdfs', debug=False)

only got sh:conforms true

With the same files, I tried TopQuadrant's SHACL API, and got the results:

<http://www.egc.org/ont/process/arcgis#project>
        <http://www.egc.org/ont/process#from>
                <http://www.egc.org/ont/process/arcgis#create_custom_geographic_transformation> .

<http://www.egc.org/ont/data#in_features_data>
        <http://www.egc.org/ont/process#isInputDataOf>
                <http://www.egc.org/ont/process/arcgis#project> .

So, how can I get the same results as the SHACL API produced ? Thanks.
The installed version of pyshacl is 0.11.4.

from pyshacl.

Hi, I am new to PySHACL but I also have the same problem that I applied :

r = validate(data_file, shacl_graph=shapes_file,
                                     data_graph_format=data_file_format,ont_graph=shapes_file,
                                     shacl_graph_format=shapes_file_format, options={"inference": "rdfs", "advanced": True})
conforms, results_graph, results_text =r
print(conforms)
print(results_graph)
print(results_text)

no inference triples are returned and the result is:

True
[a rdfg:Graph;rdflib:storage [a rdflib:Store;rdfs:label 'Memory']].
Validation Report
Conforms: True

How can I get the inferred triples? Thank you in advance!

from pyshacl.

Hi @YZhengaa,

Your example is invalid. You cannot pass "options" parameter into the validate() function. The example above is showing how to use the options when building your own Validator class instance, that is not applicable for your case.

For your example, you should use:
validate(data_graph=data_file, shacl_graph=shapes_file, ont_graph=shapes_file, advanced=True, inference='rdfs')

HOWEVER!
This still does not get your access to the new triples generated by SHACL-Rules.

PySHACL is a graph validation engine, not a general-purpose inference engine. PySHACL is not designed to provide access to the triples inferred internally by SHACL-Rules. PySHACL supports SHACL-Rules as a preliminary step in the graph validation process, and inferred triples are only for the purposes of graph validation.

Please see this Issue Thread for discussion around a potential alternate operating mode for PyShacl, which could allow the application to work as a general-purpose inferencing tool supporting SHACL-Rules.

from pyshacl.