Add `union-default-graph` parameter about oxigraph HOT 9 CLOSED

Hi! Thank you for trying Oxigraph.

By default the SPARQL queries target only the default graph. It is why SELECT * WHERE { ?s ?p ?o } LIMIT 10 returns nothing because with http POST http://localhost:8878/store 'Content-Type:application/n-triples' < ./example.nt you create a new named graph, here named http://localhost:8878/store/6368cd9e9885cb8fb9e127437a3f61af.

To add data to the default graph you should run http POST http://localhost:8878/store?default 'Content-Type:application/n-triples' < ./example.nt according to the SPARQL graph store specification.

from oxigraph.

Great, thanks so much. I just tested those commands and they work perfectly.

I still have an issue with querying named graphs. I tried various combinations of the following, but couldn't get any other response than the empty one:

• Querying the endpoint http://localhost:8878/store?graph=http://example.com vs http://localhost:8878/store
• adding a FROM NAMED <http://example.com clause in the query
• changing the query to SELECT * FROM NAMED <http://example.com> { GRAPH ?g { ?s ?p ?o } } LIMIT 10

This is probably a question that you have a quick answer for. If things get too complicated I guess I could always use the default graph.

I also know #10 added support for querying the union of named graphs, which is very useful. Does the HTTP API expose that option?

Thanks for your responses and help with this.

$ http POST 'http://localhost:8878/store?graph=http://example.com' 'Content-Type:application/n-triples' < ./example.nt 
HTTP/1.1 201 Created
content-length: 0
date: Fri, 09 Apr 2021 13:33:13 GMT
server: Oxigraph/0.2.2

$ http GET 'http://localhost:8878/store?graph=http://example.com'  'Accept: application/n-triples' 
HTTP/1.1 200 OK
content-length: 723
content-type: application/n-triples
date: Fri, 09 Apr 2021 13:35:04 GMT
server: Oxigraph/0.2.2

<http://example.org/#spiderman> <http://example.org/text> "This is a multi-line\nliteral with many quotes (\"\"\"\"\")\nand two apostrophes ('')." .
<http://en.wikipedia.org/wiki/Helium> <http://example.org/elements/specificGravity> "0.0001663"^^<http://www.w3.org/2001/XMLSchema#double> .
<http://en.wikipedia.org/wiki/Helium> <http://example.org/elements/atomicNumber> "2"^^<http://www.w3.org/2001/XMLSchema#integer> .
<http://example.org/show/218> <http://example.org/show/localName> "That Seventies Show"@en .
<http://example.org/show/218> <http://example.org/show/localName> "Cette Série des Années Septante"@fr-be .
<http://example.org/show/218> <http://www.w3.org/2000/01/rdf-schema#label> "That Seventies Show" .


$ echo 'SELECT * WHERE { ?s ?p ?o } LIMIT 10' | http -v POST 'http://localhost:8878/query?graph=http://example.com' 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query'
POST /query?graph=http://example.com HTTP/1.1
Accept: application/sparql-results+json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 37
Content-Type: application/sparql-query
Host: localhost:8878
User-Agent: HTTPie/2.3.0

SELECT * WHERE { ?s ?p ?o } LIMIT 10


HTTP/1.1 200 OK
content-length: 57
content-type: application/sparql-results+json
date: Fri, 09 Apr 2021 13:35:40 GMT
server: Oxigraph/0.2.2

{
    "head": {
        "vars": [
            "o",
            "p",
            "s"
        ]
    },
    "results": {
        "bindings": []
    }
}


$ echo 'SELECT * FROM NAMED <http://example.com> WHERE { ?s ?p ?o } LIMIT 10' | http -v POST 'http://localhost:8878/query?graph=http://example.com' 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query'
POST /query?graph=http://example.com HTTP/1.1
Accept: application/sparql-results+json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 69
Content-Type: application/sparql-query
Host: localhost:8878
User-Agent: HTTPie/2.3.0

SELECT * FROM NAMED <http://example.com> WHERE { ?s ?p ?o } LIMIT 10


HTTP/1.1 200 OK
content-length: 57
content-type: application/sparql-results+json
date: Fri, 09 Apr 2021 13:36:03 GMT
server: Oxigraph/0.2.2

{
    "head": {
        "vars": [
            "o",
            "p",
            "s"
        ]
    },
    "results": {
        "bindings": []
    }
}

$ echo 'SELECT * FROM NAMED <http://example.com> WHERE { ?s ?p ?o } LIMIT 10' | http -v POST 'http://localhost:8878/query' 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query'
POST /query HTTP/1.1
Accept: application/sparql-results+json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 69
Content-Type: application/sparql-query
Host: localhost:8878
User-Agent: HTTPie/2.3.0

SELECT * FROM NAMED <http://example.com> WHERE { ?s ?p ?o } LIMIT 10


HTTP/1.1 200 OK
content-length: 57
content-type: application/sparql-results+json
date: Fri, 09 Apr 2021 13:38:33 GMT
server: Oxigraph/0.2.2

{
    "head": {
        "vars": [
            "o",
            "p",
            "s"
        ]
    },
    "results": {
        "bindings": []
    }
}

$ echo 'SELECT * FROM NAMED <http://example.com> { GRAPH ?g { ?s ?p ?o } } LIMIT 10' | http -v POST http://localhost:8878/query 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query'
POST /query HTTP/1.1
Accept: application/sparql-results+json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 76
Content-Type: application/sparql-query
Host: localhost:8878
User-Agent: HTTPie/2.3.0

SELECT * FROM NAMED <http://example.com> { GRAPH ?g { ?s ?p ?o } } LIMIT 10


HTTP/1.1 200 OK
content-length: 61
content-type: application/sparql-results+json
date: Fri, 09 Apr 2021 13:43:45 GMT
server: Oxigraph/0.2.2

{
    "head": {
        "vars": [
            "g",
            "o",
            "p",
            "s"
        ]
    },
    "results": {
        "bindings": []
    }
}

$ echo 'SELECT * FROM NAMED <http://example.com> { GRAPH ?g { ?s ?p ?o } } LIMIT 10' | http -v POST 'http://localhost:8878/query?graph=http://example.com' 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query'
POST /query?graph=http://example.com HTTP/1.1
Accept: application/sparql-results+json
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 76
Content-Type: application/sparql-query
Host: localhost:8878
User-Agent: HTTPie/2.3.0

SELECT * FROM NAMED <http://example.com> { GRAPH ?g { ?s ?p ?o } } LIMIT 10


HTTP/1.1 200 OK
content-length: 61
content-type: application/sparql-results+json
date: Fri, 09 Apr 2021 13:46:49 GMT
server: Oxigraph/0.2.2

{
    "head": {
        "vars": [
            "g",
            "o",
            "p",
            "s"
        ]
    },
    "results": {
        "bindings": []
    }
}

from oxigraph.

Thank you for keeping to try even with all these bugs!

It seems that at some point in the store code http://example.com gets normalized as http://example.com/ because of the use of the Url library but this normalization is not done in the SPARQL query evaluation code. I have moved the code to use the oxiri library that should work better.

The SPARQL protocol specification defines that the default-graph-uri parameter instead of the graph parameter to specify the query default graph(s).
So echo 'SELECT * WHERE { ?s ?p ?o } LIMIT 10' | http -v POST 'http://localhost:8878/query?default-graph-uri=http://example.com/' 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query' should work better if there were not the URI normalization bug.

from oxigraph.

Ok, that works, thanks a lot for investigating this!

I know you're busy so I'll just mention these other things, but they are less important:

• I also tried the query changing default-graph-uri to named-graph-uri and adding a FROM NAMED clause but it didn't return anything.
• Could we update the docs to give some examples of querying named graphs?
• Would it be possible to have an option like default-graph-uri=all to use the union of all the named graphs when doing a SPARQL HTTP request, rather than having to list all the graphs to include. I know this is kind of off-spec, but other SPARQL DBs have similar options. The might need more discussion, maybe should be a separate issue.

I'd be happy to submit a PR to update that server README within the next few days.

Thanks again,
Alex

from oxigraph.

Ok, that works, thanks a lot for investigating this!

Great!

I know you're busy so I'll just mention these other things, but they are less important:

Thank you so much for taking the time to report this bug. Building a system that works well is still much more efficient use of my time wise than having a system that barely works even if it takes me more time.

I also tried the query changing default-graph-uri to named-graph-uri and adding a FROM NAMED clause but it didn't return anything.

I tried to reproduce this problem with echo 'SELECT * WHERE { GRAPH <http://example.com/> { ?s ?p ?o } } LIMIT 10' | http -v POST 'http://localhost:8878/query?named-graph-uri=http://example.com/' 'Accept: application/sparql-results+json' 'Content-Type: application/sparql-query'. It works for me. Have you checked that the trailing / are included? (if you are still using Oxigraph v0.2.2)

Would it be possible to have an option like default-graph-uri=all to use the union of all the named graphs when doing a SPARQL HTTP request. I know this is kind of off-spec, but other SPARQL DBs have similar options.

It's a great idea! Oxigraph library already provides this feature. I could very easily implement it in a few minutes. I am still unsure about which "magic value" to use: all? *? Do you have a list of systems already implementing this feature I could refer to?

Could we update the docs to give some examples of querying named graphs?
I'd be happy to submit a PR to update that server README within the next few days.

It would be amazing! I am very bad at writing documentation. Thank you!

from oxigraph.

So I had SELECT * FROM NAMED <http://example.com/> WHERE { ?s ?p ?o } LIMIT 10 instead of SELECT * FROM NAMED <http://example.com/> WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 10, which I've tested and works.

I realized I didn't really quite know what FROM NAMED did exactly and tried to find a good resource. This post describes how processors handle named graphs, and it really helped me out.

Other implementations that handle default graphs differently are Ontotext GraphDB, which makes the active/default graph the union of the default graph and all named graphs in the database

That page notes:

The SPARQL specification does not define what happens when no FROM or FROM NAMED clauses are present in a query, i.e., it does not define how a SPARQL processor should behave when no dataset is defined. In this situation, implementations are free to construct the default dataset as necessary.

Apache Jena and TDB have several seems to indicate that it's up to the user to determine what goes in the default graph pages of docs on this. The TLDR is that there are special graph names urn:x-arq:UnionGraph (all the named graphs specified in query) and urn:x-arq:DefaultGraph

I'll also link to that section of the SPARQL spec, a post about querying all graphs with a graph pattern. (I've included a lot of links, probably the most important one is just the first one)

So the decisions are:

1. What should the default behavior be when no FROM or FROM NAMED clauses are present in a query: e.g. just query the default graph as currently, or query the union of all the named graphs like OntoText?
2. Should we add an option to make the active graph the union of all the graphs?

At this point, my inclination is not to do the second option because it would be kind of non-standard and users can just do 'SELECT * WHERE { GRAPH ?g { ?s ?p ?o } } LIMIT 10'. I guess we should just decide on 1 and then document it.

We may also want to add some integration tests to make sure that queries are handled according to the precedence outlined in the first blog post linked.

from oxigraph.

Thank you for your detailed reply!

I agree that these are the two decisions that needs to be made. For 1. I would be more in favor of keeping the current behavior of only querying the default graph because it's easy to switch to the "union graph" by using GRAPH ?g { ... and the inverse is not true (there is no GRAPH DEFAULT { or similar in SPARQL query specification even if there is a proposal to add one).

About 2 I believe it is good to add an option for querying the union graph from the SPARQL HTTP protocol. This way if a user want to execute queries that targets the union graph they would not have to tweak the query itself. I already use this feature from the python API. This option would be only taken for input by the HTTP API and not expose anywhere else so it should be forward compatible with specification changes. I am thinking of using the valueless query parameter union-default-graph. What do you think about it?

from oxigraph.

That sounds great!

from oxigraph.

It's now implemented and released as part of Oxigraph v0.2.4.

from oxigraph.