Make Turtle support optional? about web-annotation HOT 22 CLOSED

👎 (against breaking compatibility with LDP)

EDIT: Many libraries exist which can be used to convert to (and from) Turtle.

from web-annotation.

@akuckartz agreed. Preference, though is for annotation to be deployed in "all" the worlds, not just the ones that can create Turtle...or even know what it is.

The core complication is that we're wanting to extend the use, creation, and distributes of annotation to places that don't know what an LDP-RS (LDP RDF Source) resource is while still allowing those who do know what they are to take advantage of all that graphy goodness. 😸

It still remains to be seen if that's even feasible...but it's the road we've taken and I'm certain others (especially other WGs at the W3C) are curious also.

from web-annotation.

From http://www.w3.org/TR/ldp/#h-ldprs-get-turtle

4.3.2.1 LDP servers MUST respond with a Turtle representation of the requested LDP-RS when the request includes an Accept header specifying text/turtle, unless HTTP content negotiation requires a different outcome [turtle].

What is the determination that a resource is an LDP-RS? Is that made by the server? Is it made by the client?

Can a "dumb" implementation of an LDP Basic Container (say a filesystem) simply send all resources in the media type they were stored with? And if it does, will LDP clients immediately jump to RDF-related conclusions if the response media type is one that encodes a graph?

Essentially, could I wrap a filesystem directory with some simple Python (implementing GET, POST, PUT, DELETE etc), store some JSON-LD for the container and update that container JSON-LD doc when a write happens, and still have an LDP client Do The Right Thing if it were pointed at this "minimal" LDP server (if it even qualifies)?

Curious @akuckartz @azaroth42 @melvincarvalho if you think that'd be feasible.

It's essentially what I started to attempt here...just using a slightly smarter "filesystem" 😉
https://github.com/BigBlueHat/ldp-on-couchdb

from web-annotation.

@BigBlueHat

I couldnt really comment on your proposal without really understanding the context better. Perhaps this note is related?

http://www.w3.org/DesignIssues/HTTPFilenameMapping.html

from web-annotation.

@melvincarvalho great read, but not quite what I was after.

Mostly what I'm wondering is how an LDP server or client expresses / "knows" a resource is an LDP-RS vs. a non-RDF Source. Is it merely that the document's media type is an encoding of an RDF-based data model (ala JSON-LD, Turtle, etc)? and, if that is the vector on which the "RDF-ness" of the resource is deteremined to what end is it valuable/important to the server or client? Can the client do anything "smart" with it--particularly in the scenario where there is only and LDP Basic Container and no Direct or Indirect access to "sub-graphs."

My feeling is that one could setup an LDP Basic Container which stores only JSON-LD has no particular graph related "smarts" and is still technically compliant per the spec. However, I'm not sure how LDP clients would feel about it. 😕

from web-annotation.

@BigBlueHat What is "smart" in that context ?

from web-annotation.

There's a nicely hidden "interaction model", which is expressed as a link header.
See section 5.2.3.4 in http://www.w3.org/TR/ldp/#ldpc-container

Also 4.4.1.2. in http://www.w3.org/TR/ldp/#ldpnr gives a link header on the response to say that the resource is not an RDFSource.

Is that what you were asking?

from web-annotation.

@akuckartz as in, understands graphs and what to do with them. 🏫

from web-annotation.

@azaroth42 the "interaction model" sounded promising for understanding this expectation, but it only resulted in using Link headers to specific the three types of containers. The question (relative to that) is what if your server only supports LDP Basic Containers, can store RDF-based things (JSON-LD), but really has no concept of what graphs are inside--hence the lack of the "smarter" Direct and Indirect Container support.

If LDP clients will still Do The Right thing with regards to CRUD on those resources, then I guess we're fine. 😃 However, if there's some other expectation here--which I'm not currently finding--then it's going to be harder / impossible to build a "dumb" LDP server--which is my aim at present. 😉

I'll certain use the LDP Non-RDF link header for things that are indeed not encodings of RDF. However, I don't want to say that about RDF-based documents (JSON-LD in this case) because they are indeed LDPRS's. Make sense? 😄 I think it's simpler than I'm making it at the end of the day. 🌆

from web-annotation.

Assuming a JSON based system, the code to produce Turtle for requests is pretty easy, even including parsing the Accept header:

def do_conneg(output):
        accept = request.headers.get('Accept', '')
        ct = default_json_content_type
        if accept:
            prefs = []
            for item in value.split(","):
                parts = item.split(";")
                main = parts.pop(0).strip()
                params = []
                q = 1.0
                for part in parts:
                    (key, value) = part.lstrip().split("=", 1)
                    key = key.strip()
                    value = value.strip().replace('"', '')
                    if key == "q":
                        q = float(value)
                    else:
                        params.append((key, value))
                prefs.append((main, dict(params), q))
            prefs.sort(lambda x, y: -cmp(x[2], y[2]))        

            format = ""
            for p in prefs:
                if self.rdflib_format_map.has_key(p[0]):
                    ct = p[0]
                    format = self.rdflib_format_map[p[0]]
                    break
                elif p[0] in ['application/json', 'application/ld+json']:
                    ct = p[0]
                    break

            if format:
                g = Graph()
                g.parse(data=output, format='json-ld')
                output = g.serialize(format=format)

        response['content_type'] = ct
        return output

So given the above 35 line implementation in python, I'm -1 to making it optional.

A second implementation (or port of the above) is sought to close the issue.

from web-annotation.

Regardless of what we say and how easy it is, if it doesn't provide a tangible benefit felt by the developer they're going to ignore it. The question for us is whether we think developers will ignore the requirement to support turtle and, if so, if we really want to say haven't implemented the protocol.

from web-annotation.

Yeah...this is where having "levels" might help out.

If my API client is in the browser, then my code is written in JavaScript and therefore the JSON(-LD) representation is what I'd be using. Even if I wanted the "smarter" graph semantics, I'd still be trafficking in JSON-LD.

The Turtle format makes sense for server-to-server or non-browser-based clients, and I'm certainly not suggesting we remove it's use altogether, just that we lower it to a SHOULD from a MUST.

If we do that, then it makes sense to also make the JSON-LD format the default and require responding to application/json (when asked)--as that too is a typical default that brings in auto-parsing, etc.

So...revising the LDP spec's 4.3.2 HTTP GET would read in our spec something like (removing the non-normative bit):

4.3.2.1 LDP servers MUST respond with a JSON-LD representation of the requested LDP-RS when the request includes an Accept header specifying application/ld+json, unless HTTP content negotiation requires a different outcome.

4.3.2.2 LDP servers SHOULD respond with a application/ld+json representation of the requested LDP-RS whenever the Accept request header is absent.

4.3.2.3 LDP servers MUST respond with a application/json representation of the requested LDP-RS when the request includes an Accept header, unless content negotiation requires a different outcome.

new bit 4.3.2.4 LDP servers SHOULD respond with a Turtle representation of the requested LDP-RS when the request includes an Accept header specifying text/turtle, unless HTTP content negotiation requires a different outcome.

(keeping the numbers from the LDP spec for reference only)

4.3.2.1 (redundant here as in the LDP spec) if you're asked for JSON-LD, send it
4.3.2.2 makes application/ld+json the default media type
4.3.2.3 makes application/json a required response option--when asked for it
4.3.2.4 (new) makes text/turtle optional, but recommended

That seems to fit the worlds of Web annotation and (typical) HTTP APIs as found "in the wild" and hopefully without being overly restrictive while still recommending support for Turtle and an understanding of the graphy goodness contained therein. 😃 That's the hope anyway.

from web-annotation.

@tilgovi:

Regardless of what we say and how easy it is, if it doesn't provide a tangible benefit felt by the developer they're going to ignore it. The question for us is whether we think developers will ignore the requirement to support turtle and, if so, if we really want to say haven't implemented the protocol.

+1 to this pragmatic view. Regardless of how easy (or not) it is to implement the json-ld to turtle conversion, we cannot ensure people will do it.

To put it more formally: if Turtle is a MUST, then a conformance requirement, as well as the corresponding tests, will have to include this. To have the protocol spec pass the bar of a Candidate Recommendation, we should have implementations really include this feature (@azaroth42, just having a separate Python helper code is not really convincing in my view, it should be part of a complete implementation). We incur the danger of not getting there.

@azaroth42:

The Python code you have shown is of course very simple at first glance. But: it relies, if I see it correctly, on the RDFLib library. Ie, the implementer has to go the extra mile to install a relatively large library which, if the implementer does not really use RDF inside, is completely useless. It is actually slightly worse: Last time I checked, RDFLib, as a default, does not include a JSON-LD parser (and serializer). An extra package (rdflib-jsonld) has to be installed. I do not see an implementer doing this just to satisfy a requirement for the standard's sake.

from web-annotation.

I do not see an implementer doing this just to satisfy a requirement for the standard's sake.

Yeah...that.

We've put some pretty serious effort into making the JSON-LD serialization of the Web Annotation model "JSON friendly." I'd like to keep that same effort in the protocol.

The higher the bar we set, the less likely as many people will (bother to) make the leap (even if they're capable of it).

from web-annotation.

@iherman: It's not a complete implementation, as I got stuck implementing paging (and hence the issues yesterday), but https://github.com/azaroth42/mangoserver is getting there as a from-scratch implementation. I'm not sure that installing two packages and then three lines of code to use them is a big deal.

Overall, I'm concerned about the tendency to favor "I don't think people would do that" over actual implementation, usage and existing standards. If a demonstration that a debated feature is easy to support is insufficient, I fail to see how we can make quantitative decisions. "Code Talks", as they say.

That said, I can live with MUST for application/ld+json (default) and application/json, SHOULD for Turtle and MAY for other RDF formats.

from web-annotation.

On 10 Sep 2015, at 17:17 , Rob Sanderson [email protected] wrote:

@iherman https://github.com/iherman: It's not a complete implementation, as I got stuck implementing paging (and hence the issues yesterday), but https://github.com/azaroth42/mangoserver https://github.com/azaroth42/mangoserver is getting there as a from-scratch implementation. I'm not sure that installing two packages and then three lines of code to use them is a big deal.

Overall, I'm concerned about the tendency to favor "I don't think people would do that" over actual implementation, usage and existing standards. If a demonstration that a debated feature is easy to support is insufficient, I fail to see how we can make quantitative decisions. "Code Talks", as they say.

That said, I can live with MUST for application/ld+json (default) and application/json, SHOULD for Turtle and MAY for other RDF formats.

So can I

Ivan Herman, W3C
Digital Publishing Lead
Home: http://www.w3.org/People/Ivan/
mobile: +31-641044153
ORCID ID: http://orcid.org/0000-0003-0782-2704

from web-annotation.

@azaroth42 I'm sympathetic. I don't like removing stuff just because some people might not implement it. So I support the SHOULD.

I'm the sort of developer that would be happy to write a few lines to appease a spec, but then I'm also the sort of developer who would participate in a WG, and that's not super common.

from web-annotation.

That said, I can live with MUST for application/ld+json (default) and application/json, SHOULD for Turtle and MAY for other RDF formats.

@iherman @azaroth42 how should we route that toward approval / consensus / integration?

It sounds like we MAY have a winner. 😉

from web-annotation.

@azaroth42 FWIW I am also an independent developer that has participated in W3C WG's. Im not employed to do so, but I like to implement things that solve problems in a practical way. Normally I will just try and choose the tool that will solve use cases in the fastest possible way. My personal preference is turtle right now, with JSON LD not far behind. I dont say this from any academic or theoretical standpoint. Just experience of trial and error.

Linked data is still really new in terms of things implemented, I think we're still around the time of geocities in web history. It can be frustrating when hearing things like 'developers will/wont do X ...' -- when actually we've never really been asked and there's not good data to back up or counter act pre conceived notions (normally from people that have never even implemented a user table using linked data). I'm very interested in the work of the WG, and would be looking to take pieces of it for a system I'm working on. I'd be really happy to use primarily turtle, secondarily JSON LD.

Just my 2 cents.

from web-annotation.

@melvincarvalho Thanks for the input, much appreciated :) Any feedback that you or others can give will only help make the result stronger, rather than built on likely incorrect assumptions.

@BigBlueHat I've added a proposed resolution to the agenda for next call. Hopefully we can knock out a reasonable set of issues, including this one.

from web-annotation.

@melvincarvalho thanks for the input! It wasn't my intent to bandwagon. The statements above are rooted in two things:

1. Linked Data tooling is (as you stated) known to be in it's formative days as yet
2. the less overhead required (for anything) the more likely it is to happen

I'm working on an LDP / Web Annotation Protocol server for CouchDB. The current attempt is to build it all within CouchDB and avoid any additional languages or run times--so if I can't code it in non-DOM enabled SpiderMonkey friendly JavaScript, it can't be done. This, afaik, is probably the most constrained environment we'll face (or that's my hope in attempting it) and anything beyond that MAY have Linked Data tooling already available.

There are plenty (as you know) JS related Linked Data things--which I went hunting for early on--but found that the JSON-LD <=> Turtle transformation world was a tangle of various librariers that seemingly do nearly the same things causing the transformation between those two formats to be several lines of code written by someone who understands that N3 sits somewhere in that mix and that there are other JSON encodings of RDF that are used by these libs as intermediaries, and... etc. In the end, I tabled that effort, and circled back on just getting the API bits implemented around JSON-LD.

Browser "level" JS developers will be using (most) of those same tools and will hit the same thing.

Could this be solved by more mature tooling? Probably.

Will developers still need to learn A New Thing to Do It Right? Definitely.

Is that a Bad Thing? Nope. 😄

It comes down to a question of how much learning should/must be required for someone to build an annotation endpoint and serve and accept incoming annotations.

That, sadly, isn't a precise measurement, but the "geocities" early days of Linked Data tooling is currently an inhibitor...at least for this developer. 😸 Despite his desire to use it... 😕

from web-annotation.

Done: http://w3c.github.io/web-annotation/protocol/wd/#annotation-retrieval

from web-annotation.