Does anyone actually use JSON Hyper-schema about json-schema-spec HOT 101 CLOSED

I use Hyper-Schema for all of my JSON based REST APIs (although I haven't had the opportunity in while). In fact, I would have very little interest in JSON Schema if it weren't for JSON Hyper-Schema. As far as I have been able to find, it is the only standard capable of doing hypermedia APIs in JSON properly (1).

In REST, resources should be self descriptive giving you everything you need to know about what it is and what you can do with it. On the web this takes the form of links and forms. Many JSON standards out there can describe links, but only JSON Hyper-Schema can do something like a form. If you are familiar with the Richardson Maturity Model (2), all the other options are designed for Level 2 REST APIs. JSON Hyper-Schema is the only Level 3 option around.

I created a little library a while back that allows you to create fully functional CRUD APIs by writing only Hyper-Schemas (3). These APIs can be used without any out-of-band knowledge using Jsonary's (4) generic Hyper-Schema browser. This library allowed me to walk through the workflow while designing APIs. Nothing else is capable of that.

This was a much more long winded response than I planned on, but my ultimate purpose is to say that I am strongly against changing the Hyper-Schema approach. There are certainly things that can be cleaned up, refactored, or extended, but I'll be keeping an eye on any Hyper-Schema proposals to ensure that the spirit of Hyper-Schema is not lost.

I'm all for the kind of changes it sounds like @awwright is talking about, but I am concerned about what I think I'm hearing from @handrews. I look forward to reading your proposals.

(1) JSON-LD has Hydra and XML has XHTML, but JSON has only JSON Hyper-Schema
(2) http://martinfowler.com/articles/richardsonMaturityModel.html
(3) https://github.com/jdesrosiers/resourceful
(4) https://github.com/jsonary-js/jsonary

from json-schema-spec.

@Relequestual, I'm a little more strict about what I consider RMM Level 3 than the general community. To explain, I'll introduce the Desrosiers Maturity Model (DMM) for Hypermedia APIs :-). Like the RMM, the lower levels incorporate only the most basic concepts. Based on the DMM scale, the general community seems to be accepting of anything Level 1 or higher, while my statements were inclusive of Level 2 or higher.

The general concept is that responses should be self-descriptive. Like an HTML browser, it should be possible to navigate a Hypermedia API without any out-of-band documentation. The levels go from more documentation needed to less.

Level 0: At this level, a representation would include URIs referencing another resource, but they are just strings with no semantics allowing a machine to recognize it as a link and processes it as such. To a computer, these URIs are just strings. Documentation is needed to know that these links should be followed. This is like not having a <a> tag in HTML. You can have the URIs text on the page, but you have to follow them manually.

{
  "id": 23,
  "title": "My Blog Post",
  "content": "...",
  "author": "/api/author/1",
  "comments": "/api/blog/23/comments"
}

Level 1: The next step is to use a media type that defines link semantics allowing a computer to recognize it as a link. This is like having an <a> tag in HTML. A user agent can now follow these links to related resources. Level 1 is sufficient for read-only APIs, but is lacking when user input is required. If you need to send data to the server you have to describe in text how it should be done. This is like not having a <form> tag in HTML. HAL is Level 1.

Level 2: To be Level 2, you need to a hypermedia format that has a feature that resembles an HTML <form>. JSON Hyper-Schema has this ability and at the time I made the claim you quoted, I had not seen another option that did. Confession time. I took another look at Collection+JSON and Siren and saw that they do have this feature after all (Collection+JSON calls them templates and Siren calls them actions). I must have missed it in my previous evaluations. So, based on my own definition, I have to include them as options for achieving RMM Level 3. I saw another media type recently that would qualify as Level 2, but I can't find it now and don't remember what it was called.

Hyper-Schema is still my heavy favorite because of it's superior ability to describe request body requirements and it's simple and unobtrusive syntax. Collection+JSON only seems to be able to declare property names of the data you want to send. Siren goes one step further by allowing to specify a datatype. Both of these options are incredibly rudimentary compared to being able to describe the request body with a JSON Schema. Hyper-Schema wins big in that respect.

Level 3: Level 2 is sufficient for most APIs, but it is possible to take things even further. Just like how we said in Level 0 that a URI is just a meaningless string to a computer, property names like "title" or "author" also have no meaning either. If your data is self-descriptive, you now have the possibility of automating things like interactions between systems where it usually requires a human to hard-code the connecting points. A form could be filled in automatically because the computer knows how the data it has matches up to the form data being requested. This can be achieved using JSON-LD in conjunction with a Level 2 hypemedia format.

There is a format called Hydra that is based in JSON-LD, but I don't think there is any reason why JSON-LD responses couldn't be used with any of the other Level 2 formats mentioned to the same effect. However, Hydra is the only hypermedia format I know of that is Level 3 without having to be combined with anything else. That said, I'm not a fan of Hydra. I found it to be way more complicated than it needs to be and soon went back to simple and unobtrusive Hyper-Schema.

So, in conclusion, I was wrong that there aren't any other options out there, but I still think JSON Hyper-Schema is easily the best option.

from json-schema-spec.

We use JSON Hyper-Schema extensively in both our platform API and client (really just JSON Schema in the back-end for validation purposes). I have worked on another enterprise API that was based on the Siren specification, but JSON Hyper-Schema is far superior due to its insane flexibility, its ability to keep things DRY and support for meta schemas ✨. I completely agree with @jdesrosiers that Hyper-Schema is the only JSON-based hypermedia API specification that is truly Restful. My favorite aspect is how it doesn't require you to modify your API responses, it's entirely non-invasive and complementary.

I'm not sure what issues you are having resolving your refs, but we have found Ajv to be very consistent and robust in resolving them. Many libraries, such as is-my-json-valid, expect the user to provide a self-managed object mapping $ref to schema.

The only "limitation" our team has encountered (really just an initial mis-understanding) has to do with entity instances that are required/used for resolving URI Templates into absolute URLs - the issue is that some APIs want to use deeply nested resources that require multiple entities, like so:

/api/v1/a/{uuid}/b/{uuid}

The questions we ran into were "Do we use the same entity instance? Another? If it's from another, how do we specify where the instance should come from?"

Our complications were mostly due to the fact that we designed our schemas to map nearly 1:1 to our API's domain model entities, such that each schema correlated to a single high-level entity (and I believe this is the general idea, but I'm sure there's other opinions on that). We still referenced other schemas with $refs and had normalized schemas, but the general idea was that the links defined in the schema only needed to know about the entity correlating to, well, that schema.

Because of this, it wasn't initially clear that support for working with multiple entity instances when resolving LDO (Link Description Object) URI templates is possible with JSON Pointer references - otherwise the URI template slugs correlate with the defacto entity instance provided (http://json-schema.org/latest/json-schema-hypermedia.html#anchor8). The question you still have to answer in your client, though, is what entity should be used when the reference is say #/foo but the user has multiple Foos to work with in the client - should it be the first, the last, the "selected" one by the user? I think it's fine to allow the application to handle this in whatever way they want, it's just not explicitly talked about in many JSON Hyper-Schema resources/tutorials.

Once our client-side API resolves the URI Template into a URL based on the provided entity instance, we then just construct an HTTP/REST resource caller that automatically follows whatever method is defined in the LDO

Edit: couple typos

from json-schema-spec.

@jdesrosiers Great feedback, thanks! As how REST has the code-on-demand constraint (which I take as including scripts and stylesheets), I sort of view JSON Schema as similar to a stylesheet, JSON Schema tells you how to render or how to work with the data you've been given.

One of my outstanding concerns with JSON Hyper-schema is it has a "method" property combined with "href" and "rel". A link is supposed to describe relationships between resources, not necessarily listing things to do. Retrieving information about an associated resource is GET, HEAD, or OPTIONS, changing a resource is PUT or PATCH, executing some remote function is POST.

Can you provide some more examples of how you'd like to approach linking?

I'm re-reviewing the Richardson Maturity Model, which is so great for describing what RESTful really means. I'm also reviewing RESTful Web APIs by Leonard Richardson & Mike Amundsen (the legends of hypermedia protocols).

from json-schema-spec.

@slurmulon We could have a field similar to HTTP "Allow" that specifies which methods the resource is capable of handling. So if "PUT" is missing, then it's a read-only resource; if POST exists, then it's an executable resource; etc.

@handrews See how many of those concerns have issues filed against them? I should create a milestone for this maybe.

from json-schema-spec.

@jdesrosiers I just read through RFC 6573: "The Item and Collection Link Relations
" and noticed that it lists application/vnd.collection+json and application/vnd.maze+xml as two examples of usage. Both of those specifications define that a "collection" link means that HTTP POST can be used to create a collection item.

So I think once Draft 06 is out the door I'll file a new issue for JSON Hyper-Schema to reference RFC 6573 and specify the semantics of those link relations in the context of application/schema+json. That seems to be both valid an useful. And would be enough information for a generic client to process correctly. An HTTP header with a rel of "collection" is not guaranteed to accept a POST to create, but we can specify that an API claiming a conforming use of Hyper-Schema MUST either do so or return a 405 Not Allowed for POST. Or maybe SHOULD. We can debate it after Draft 06 is out.

The general issue of overly broad link relations is still an issue, but it seems like this case seems easy enough to solve. This would unambiguously replace Draft 04's proposed "instances" and "create" link relations with existing IANA-registered link relations.

from json-schema-spec.

@Relequestual FWIW I think that we should also have keywords that are documentation-oriented. I don't think that static documentation and dynamic hypermedia are incompatible, they're just different things that need different support. For instance, #73 is a way to support static documentation, but it did not get momentum for Draft 06 (and see also mnot/I-D#211 for more discussion related to the topic of HTTP hints).

from json-schema-spec.

@jdesrosiers while I would happily reboot hyper-schema, a more incremental approach based on adding rather than changing is entirely possible.

Honestly, I've been saying provocative things about hyper-schema mostly to motivate people to actually speak up. "Resourceful" looks really interesting, I'll have to look into it more deeply. Part of my interests here have been around seeing if anyone has developed best practices for hyper-schema that mitigate the problems we found. If so, I am absolutely not dead-set on changing it.

I agree that JSON Hyper-Schema is the only thing that comes close to RMM Level 3. It is actually what helped me sell RMM Level 3 at Riverbed, but ultimately the team found hyper-schema too confusing to accept.

At a high-level there were three problems, ranging from annoyances to deal-breakers. I was going to wait and file proposed solutions (um... after I figured them out), but since we're seeing some life here, let's just look at the problems. Perhaps someone else figured out a way around these, or can explain what we missed along the way.

This is, of course, my personal opinion and recollection several years down the road, and not the official opinion of my employer at the time (or current employer).

Implicitly defined resources

Hyper-Schema implicitly defines resources by defining links to them. Every schema that has a self link is a resource (relatively explicit), and every link URI that is not otherwise accounted for in a self link is also a resource (it can be confusing to line these all up).

This means there is no straightforward listing of resources anywhere obvious. Unless every resource is accessible from the root, which is not the case in many non-trivial APIs. It also means that with link URI templates defined all over the place, it is easy to produce conflicts, which also plays into the next problem.

This is the problem that only some people considered a deal-breaker. I was not one of them, but there were very strong opinions about this. Our solution was to separate the declaration of resources and links from the schemas. Schemas could be defined inline within the resources, but usually were $ref'd in.

I would like to sort out best practices for making the resources more clear, but am not at all attached to the separate enumeration approach. I can see "fixing" this just by implementing the documentation generator in a way that pulls the resources out and enumerates them clearly in the docs.

Repetition in link definitions

There's a lot of repetition in link definitions, or else functionality left out that should be doable in one step. Simply put, if I can do all the usual CRUD stuff on books and people, and a book has a list of authors, all of the links I define on authors are redundant. They're already defined for people. But I have to re-define them with a slightly different URI template of "/people/{authorId}" instead of "/people/{id}", and re-define the schema non-authoritatively in targetSchema (which can at least be done with $ref). I either need to re-define every person link, or force the caller to first GET the author in order to discover that it can be DELETEd. However, many operations, including DELETE, should not require a GET first, so that's back to duplication.

What makes more sense when both resources are described in the schema is is to be able to define a relationship between books and people with a rel of "author" and rules for how to translate the book instance data into a form that can fill out the self link for "person." e.g. map "author_id" in book to "id" in person (assuming person's self link is something like "/people/{id}").

This approach doesn't technically make anything impossible, but it was considered excessively burdensome. The only way we would have been able to use JSON Hyper-Schema would have been to write a pre-processor that duplicated all of the links out to the proper places.

Our solution to this was just to add a "relation" field that referenced the other resource (it helped to have resources enumerated separately here, but is by no means required), and provided a map from the source schema's fields to the destination schema's self link URI template. This was done by the vars approach seen in issue #52 (extended href templating). Using the same vars mechanism in multiple places made things a lot easier- it is always the way to translate URI parameters when they don't line up perfectly with the URI template syntax.

[edit- removed the bit about href duplication from self as Hyper-Schema actually does that part fine and I just forgot]

Another benefit of this solution was that it removed the need for most targetSchema definitions. Instead, you just looked at the target resource's self link. The relevant security considerations are already described for self links. targetSchema is still available for when you want to link to something outside of JSON Hyper-Schema.

I like this solution very much, but would be happy with anything that satisfied the same requirements of eliminating the duplication and more clearly showing resource-to-resource relations.

Can't describe both URI parameters and a message body

This makes JSON Hyper-Schema unusable for me. There is only one schema in the LDO, which applies to the URI parameters for GET and the body for POST (and presumably for PUT and PATCH although they are not mentioned?). Unless I am totally misreading things (which I could be), this makes it impossible to perform operations with a message body on URIs with query parameters. This means no such operations on filtered collections (a powerful alternative to messy batch processors), or partial representations (selecting a lighter-weight format), or anything else involving non-hierarchical identifiers.

Our solution is the one described in issue #52 (extended href templating). We used URI templates for all query string parameters. URI template variables either mapped to points in the instance schema (by the default action or through vars), indicating that they used the same validation rules, or they were given a schema directly if they were parameters that must come from an external source.

The extended href templating proposal was already in existence, we just adapted it and standardized that the request schema never applies to the URI, it only applies to the request body. Again, without a solution for this, JSON Hyper-Schema cannot adequately describe my APIs.

from json-schema-spec.

I was going to close this, I forget why I didn't, I think I was waiting on something. But this is answered to my satisfaction now.

from json-schema-spec.

@Relequestual @jdesrosiers I'm part of the way through a big comparison of hypermedia vs API description which I'll put on the wiki. Or maybe our web site through the other repo. I just got distracted replying to #280 so it will have to wait until tomorrow :-)

from json-schema-spec.

From my point of view, I'd be happy to reboot the whole hyper-schema approach while preserving some specific elements. In my past work, we picked bits and pieces from JSON Hyper-Schema to assemble into an alternate approach that got broad buy-in where Hyper-Schema as defined did not.

That has a bit more to do with what Hyper-Schema should look like in v6, but it does indicate (for me, at least) that the v5 cleanup can be pretty aggressive since we should expect significant changes for v6 anyway.

from json-schema-spec.

@awwright I use it for defining a link with rel of "options" to then pull in the options for a select.

Totally agree that it doesn't seem to be the best spec of the bunch and more than happy to re-work if there are changes. But just wanted to submit a use case for consideration.

from json-schema-spec.

@awwright : your comment here

One of my outstanding concerns with JSON Hyper-schema is it has a "method" property combined with "href" and "rel". A link is supposed to describe relationships between resources, not necessarily listing things to do. Retrieving information about an associated resource is GET, HEAD, or OPTIONS, changing a resource is PUT or PATCH, executing some remote function is POST.

captures my key concern in 500% less words :-)

Basically, resources are not a first-class concept in JSON Hyper-Schema. They kinda just happen, and a lot of weirdness exists around that.

from json-schema-spec.

@slurmulon I'm looking at "rel"... the current draft wants to add four values to the IANA registry and I'm not sure all those are necessary.

As for nested resources, unfortunately I'm not really sure how to handle the more complex URI cases.

Consider, though, if you're listing an absolute URI in a URI Template, then you're sandboxing yourself inside that namespace. It means if I want to store instances of resources on my server, I have to modify the schema so it points to my server instead of yours.

So increasingly, I think the best design pattern is that JSON documents should provide pre-computed absolute URIs or URI references, and minimize computation. The only time I would use an absolute URI in a URI Template would be if it's globally assignable anyways, like "urn:uuid:{uuid}", "ni:///sha-256;{hash_sha_256}" and the like.

The other alternative is we provide some way to let a script compute a value on a JSON instance, including provide a list of link relations. This is certainly within the scope of REST, but it seems excessive.

from json-schema-spec.

@awwright : We set a base URI for all of our schemas and then if the template began with $, the dollar sign was replaced with the base URI. This is similar to issue #46 (LDO baseUri) but at the scope of an entire API (a concept which JSON schema also lacks, although making a schema that is all definitions plus a schema for the entry point resource can more or less serve that purpose).

Anyway, this removed all need for absolute URIs for us except when going outside of the API entirely, in which case it was fine. We needed to be able to define URIs relative to the API base without having to repeat the API base (e.g. "/some/api/base/path" in "https://api.example.com/some/api/base/path") all over everything.

from json-schema-spec.

@awwright Regarding nested resources and how they are currently handled, I stumbled across something like this (yes, with encoding):

/api/v1/user/{ root.json%23%2Fdefinitions%2Fuser }/quotes/{ root.json%23%2Fdefinitions%2Fquote }

I'm unable to find the resource for this, I ran into it months ago and it's very possible that it isn't officially supported (not seeing anything mentioned here: https://github.com/json-schema/json-schema/wiki/href or in the spec). Anyways, it's the closest thing I could find that would allow us to support multiple entity instances (our API team was annoyed with this fact, but I don't think it's a big deal at all). We had to write our own LDO href resolver and we just didn't add support for referring to other entities, particularly because of the complexities around "which one is the right entity?" The schemas define what an entity can be, not which one it is - and in my opinion this is a very good thing, it's just something for the client to consider in its state management/design.

I agree that you should minimize computation with your links, it will generally make everybody's life easier. I think though that it will still be necessary, even for common cases such as /api/v1/user/{uuid} - this prevents the need to modify the API response in any way (:heart:) - other hypermedia specs like Siren do not have this luxury. This strict separation also ensures that the API never needs to concern itself with any sort of state (specifically to answer "which entities should be used to make this link?"), satisfying HTTP and REST.

Even with globally unique identifiers, you still need to ask the question of "which instance do I use for the other entity/entities?" in the case of LDOs with more than one entity reference. With one entity it's trivial, because you can always just associate the schema/schema validator with whatever entity you want. But when there is a nested entity, it must either also be explicitly provided by the dev (increasing domain coupling, nipping away at a clean design) or implicitly provided by some state management system, and that (to me) is the hairy part. But again, I think this is primarily a concern of layers above JSON Hyper-Schema.

Edit: clarity, grammar

from json-schema-spec.

@handrews

Basically, resources are not a first-class concept in JSON Hyper-Schema. They kinda just happen, and a lot of weirdness exists around that.

I've noticed this as well, what do you think can be done to help alleviate/fix this? What if you say that an LDO method correlates with the resource's OPTIONS when the value is an Array (please excuse the rel here, clearly not sufficient):

{
  "rel": "self",
  "method": "GET",
  "href": "/v1/api/user"
}
{
  "rel": "updates",
  "method": ["POST", "DELETE"],
  "href": "/v1/api/user/{uuid}"
}

Otherwise, if it's a single value, method should be interpreted as the method that should be used (in other words there is only one "usable" method on the resource, so there's no point to provide a collection).

It seems this could help to eliminate redundancy in the LDOs as well since there's no need to duplicate links for additional actions that can be made on a resource. Perhaps the example I provided could be collapsed even further with URI Templates, such that GET, POST and DELETE can exist in the same LDO.

from json-schema-spec.

@awwright yeah love it, that way method doesn't have two different meanings depending on its value

from json-schema-spec.

Unlike jsonschema-core and jsonschema-validation, I'm at something of a loss for how to save jsonschema-hyperschema.

My best idea right now is I'll rewrite major sections of the spec, and won't be afraid to break reverse compatibility with existing implementations. These are just Internet-Drafts, after all, and I think this is warranted because HTML, Atom, HTTP Link, HAL, etc, have much bigger adoption and are far more consistent, and have probably seen better adoption because of their consistency.

These are the following features I'm looking to gently change or preserve, everything else I'll tear out unless someone says they'd like it:

• links (how to extract hyperlinks - list of Link Description Objects)
  • anchor (e.g. "http://example.com/subject")
  • rel (e.g. "self")
  • rev (reverse link, e.g. "author")
  • href (e.g. "http://example.com/object")
  • hreflang (e.g. "en")
  • media (e.g. "screen" "print")
  • title (e.g. "Measurement Levels and Configuration: A Linear Approach")
  • type/mediaType (e.g. "application/json")
  • accept/methods (e.g. "GET, HEAD, OPTIONS, POST, PUT, PATCH")
  • schema
  • enctype
  • schema
  • targetSchema
• media (how to interpret a string as an information resource)
• readOnly (the instance is managed exclusively by the authority, don't bother changing it)

During my survey, I found or came up with these features that I think would be useful:

• deprecation (provides information that an instance shouldn't be used and might be removed in the future)
• base (changes the URI base of an instance to this value, after URI template processing)
• forProperties, forItems, forAdditionalProperties, forAdditionalItems (descend into an array or object, but links still describe the current JSON instance)

I'll create issues for these enhancements.

Please let me know all your opinions!

from json-schema-spec.

@awwright Wow, I did not know that Hyper-Schema was in such a dire place - I have experience with HAL, JSON-LD, Siren, and more, and JSON Hyper-Schema is by far the most extensible and non-invasive hypermedia spec I have ever encountered. Its ability to represent and work with highly-complex data structures is simply unparalleled, and the fact that it establishes a unified validation language between layers of the stack is absolutely incredible (and generally long desired!) as it completely prevents any sort of code duplication or parallel business logic across layers. Its integration with things like API Blueprint allows you rapidly accelerate productivity with tools like Drakov (mock server) and Dredd (test your API Blueprint, and the JSON Schemas defined in it, against a real API). These are all amazingly helpful features that other specifications simply don't have, don't allow, or struggle with greatly.

I'm happy to do whatever I can to help keep the spirit alive (documentation, technical/design discussions, whatever). I believe a lot of this stems from a lack of documentation / examples that give people their "Aha!" moment. It took our team a minute to realize the magic of Hyper-Schema, but once our team did it was a no-brainer as to how much it could help, and it did.

As for the feature list you mentioned, I love the idea of a base URL/URI, this would help relieve a lot of the complications regarding $ref resolution and JSON Pointer

As for readOnly, wouldn't you just implicitly know that the resource is read only when the only available accept / methods is ['GET']? Perhaps that's too indirect, but could save the need for an extra property.

Regarding title - is this intended to be used as a presentation label for the link? IME things kind of break down once you get too far down HATEOAS (i.e. Browser in a Browser) because the server is now concerned with UI presentation, and it also complicates i18n/l10n/partnerification if say the text translations are handled on the client side, which is relatively common. Perhaps in this case gettext tokens can just be sent in title and the client would be expected to replace them, not bad I guess (thinking it's probably best to make this optional..?)

I really love the idea of "forProperties, forItems, forAdditionalProperties, forAdditionalItems (descend into an array or object, but links still describe the current JSON instance)" - this would allow semantic iteration through specific areas of an instance object, which is great!

This wasn't mentioned, but I like the idea of adding $ref to LDOs (in addition to targetSchema, because to my understanding this only applies to the API response instead of the request) since it would allow the client to pre-validate an object before sending it off to the API, allowing the client to save an extra network call.

from json-schema-spec.

@awwright @handrews I would love to join the organization if you're accepting members. I use JSON Hyper-Schema in many of my projects (including enterprise ones), so I am very passionate about the role that it can play in the Hypermedia world and have a lot of incentive to keep the ball rolling.

Please feel free to email me at [email protected] if you'd like to discuss this further!

from json-schema-spec.

@slurmulon I don't have any official position here myself, I'm just talkative and opinionated :-)

from json-schema-spec.

@slurmulon Luckily I don't see the spirit of hyperschema changing at all, I think we're just realizing there's a small handful of things that don't mesh well with current hypermedia design best practices.

title is a standard link parameter, RFC5988 says:

The "title" parameter, when present, is used to label the destination
of a link such that it can be used as a human-readable identifier
(e.g., a menu entry) in the language indicated by the Content-
Language header (if present). The "title" parameter MUST NOT appear
more than once in a given link-value; occurrences after the first
MUST be ignored by parsers.

readOnly is mostly for individual (sub-)instances, so you can say particular properties in an object are managed by the authority (the server). Things like serial numbers and computed values, where it only makes sense for the server to assign or update and not anyone else.

Regarding $ref in LDO, can you file a new issue that includes some examples and use cases?

from json-schema-spec.

@slurmulon and @awwright : I'm extremely confused by both of these things:

forProperties, forItems, forAdditionalProperties, forAdditionalItems (descend into an array or object, but links still describe the current JSON instance)

/api/v1/user/{ root.json%23%2Fdefinitions%2Fuser }/quotes/{ root.json%23%2Fdefinitions%2Fquote }

Are these attempts to use parts of the instance data other than the immediate properties for resolving the URI template variables? Or for using instance data from within elements of a list or properties of an object within an instance?

If so, those problems are solved very cleanly by the advanced templating proposed in in issue #52 which we used extensively throughout a set of APIs involving about 200 or so resources. I'm honestly baffled at the lack of reaction to that proposal, as it's the single most essential proposal outstanding for hyperschema for me (among other things, it is necessary to solve the failure to handle both URI parameters and message bodies, and (with relative JSON pointers) it handles URI parameters far more flexibly and elegantly than anything else I've seen proposed- no awful %37elf mess.

from json-schema-spec.

@awwright yeah "spirit" probably wasn't the best word choice - I guess I meant "momentum" since the title of this issue is "Does anyone actually use JSON Hyper-Schema?" hehe

title is a standard link parameter, RFC5988 says:

Cool, that makes sense. I was making a wrongful assumption of how the value would/could be used.

readOnly sounds great as well - is this expected to be a simple boolean or say a collection of sub-paths (i.e. JSON Pointer) to mark as read-only?

More than happy to file an issue that includes some examples and use cases. Stepping out for a few hours right now but will get to it later tonight/

from json-schema-spec.

@slurmulon You put readOnly throughout your schema as a boolean on properties/items/whatever that need to be readOnly. We used this a lot at Riverbed, very useful (I can point you to public example schemas if you'd like to see how we used it).

from json-schema-spec.

@handrews regarding the work you did for #52 looks extremely useful and elegant - speaking for my own example, yes, it's an attempt to use parts of the instance data to resolve URI Templates (and instance data that potentially correlates to another schema, which raises the difficult question of "which entity instance should I use?"). I'm happy to help create examples, justifications, whatever, if you think that might help get things moving

I would like to see how you are using readOnly, we really hadn't made that consideration in our platform and our API sort of just ignores setting read-only values like UUID, so I'd love to see how we can stop doing that and instead leverage JSON Hyper-Schema

from json-schema-spec.

I was hesitant to comment on this issue because I don't have the time to adequately keep up with the discussion. So, I'm sorry, I won't be able to answer all of the questions or comment on all of points made (at least not in a reasonable timeframe). For now, the following is how I think about Hyper-Schema in response to @awwright's inquires. As for specific difficulties with Hyper-Schema, @handrews, I'll try to address that at a later time if I can.

When thinking about REST architectures, I find it useful to make analogies to it's reference implementation, the web. However, it is important to remember that the web doesn't define REST, REST describes the web (for the most part). So, the analogy is not always perfect.

A JSON document is analogous to an HTML document. A Hyper-Schema can be linked in much the same way as a stylesheet, javascript, or image to completely describe the resource. This is where Hyper-Schema most deviates from the HTML analogy. It would be like if all the links and forms in your HTML were defined in a separate resource. It's different, but I've tentatively convinced myself that it doesn't violate any REST principles and it definitely has it's advantages. To make the analogy more clear, it helps to think of the analogy to HTML as the JSON document + the Hyper-Schema.

The next part of the analogy is hyperlinks like the anchor tag <a\>. This is the thing that most hypermedia offerings are designed to describe. @awwright, I think you are viewing Hyper-Schema links (LDOs) from this lens and thus are under the impression that they are doing too much. Although LDOs are capable of describing a hyperlink, they are not an analogy for hyperlinks.

LDOs are an analogy to an HTML <form\>. All of the things that you recognize as not belonging as part of a hyperlink, do belong in a form. There is not a hyperlink analogy in Hyper-Schema because the functionality of a form subsumes the functionality of a hyperlink. Therefore, it was unnecessary to describe them separately. This form-like functionality is the killer feature of Hyper-Schema that no one else has. I have to be careful with this part of the analogy because you shouldn't take this to mean that you necessarily should be able to generate a pretty UI for an LDO. This is the machine-to-machine version of a form.

One last analogy is Web 1.0 vs Web 2.0. Web 1.0 describes a web that is mostly static. It is mostly driven by hyperlinks and there are very few opportunities for users to interact or contribute content. Web 2.0 describes a web that is dynamic and makes heavy use of forms to interact with users. Other hypermedia solutions are capable of describing Web 1.0 style APIs were resources are largely read-only. Hyper-Schema allows a Web 2.0 style API that is naturally interactive. By naturally, I mean that there is no out-of-band information necessary to use it. That means a Hyper-Schema driven API needs no documentation to use it in the same way a website needs no documentation. Everything you need to know to use the website is included in each request.

Ok, I lied, I do have one more analogy. targetSchema was mentioned a few times. If I have my way, this keyword would be removed entirely. I know it is defined to be non-authoritative, but I think it's presence just enables and confuses people who still have not completely gotten away from the RPC mindset. When you click a link on a webpage, other than the URI, you don't get a guarantee or even a hint at what you are going to get. Whatever resource you end up with tell you what kind of resource it is and what you can do with it. You don't need to know ahead of time. This is one of the oldest concepts baked into the web and it is one of the things that has allowed it to scale the way it has.

One last thing, @handrews, I don't understand your position that Hyper-Schema is not resource-oriented. All interactions are done through interacting with a resource. The resource itself describes what you can do with it. What could possibly be more resource-oriented?

from json-schema-spec.

@slurmulon : For the examples, I'll send you info offlist- anyone who also wants info just comment or or start a thread on the Google Group. If anything interesting comes out of offlist discussions I'll post a summary back here.

I am a bit confused still about "instance data that potentially correlates to another schema". Do you mean the schema for a different resource, or do you mean a parent or child schema of the one defining the link?

To give credit where credit is due, the extended templating proposal was originally from Geraint Luff. We built on that with the "relations between resources" concept I mentioned earlier, but it was mostly already proposed.

from json-schema-spec.

@handrews thanks for adding me to the Google Group, sounds good.

Regarding "instance data that potentially correlates to another schema", consider this URI template:

/api/v1/user/{uuid}/quote/{uuid}

If we only had /api/v1/user/{uuid}, it's completely trivial which object we want to pluck uuid from - a valid user object that's being provided to whatever method resolves our LDO URI templates.

When there is more than one identifier slug, such as with /api/v1/user/{uuid}/quote/{uuid}, it now becomes ambiguous which schema or entity instance object uuid is supposed to map to (#/user or... what?). The only time you don't have to ask this question is if the user response always includes a denormalized quote, because then you could just do something like this:

/v1/user/{uuid}/quote/{quote.uuid}/

But if you want to keep your schemas and API resources relatively normalized (which I generally do, not sure if most people feel this way), nested identifiers introduce a problem. This is why the syntax I mentioned above, using the encoded JSON Pointers, answers this question at least partially, the question of what schema should this URI Template slug correlate to. It does not, however, answer the question of which entity instance to use - this needs to be answered by the developer for sure, but the question becomes much harder when there are two entities to juggle instead of one in the client.

from json-schema-spec.

@jdesrosiers : Thanks for taking the time to put that together. I have several responses :-)

This is the machine-to-machine version of a form.
Yes! This. Absolutely.

I don't understand your position that Hyper-Schema is not resource-oriented. All interactions are done through interacting with a resource. The resource itself describes what you can do with it. What could possibly be more resource-oriented?

I mean that resources are not given clear visibility in a JSON Hyper-Schema based API. If I want to pull out a list of resources for documentation and determine which of those resources are related to each other in some way, I have to:

1. Go through and find all URI templates, keeping track of the relation name and the schema to which they were attached.
2. Any URI template that is a "self" link defines a resource described by the schema in which it is listed as "self"
3. Group duplicate URI templates, which may be tricky to determine as Hyper-Schema requires duplicating URI templates with slightly different variable names under certain circumstances
4. Any non-"self" URI template that is the duplicate of a "self" URI template indicates that there is a relationship (or a set of relationships, if the LDO is defined on array items) between the resource with the non-"self" link and the resource with the matching "self" link.

Whew. That's a pain in the ass to even think about, much less code. I'm not even 100% sure that step 3 is entirely possible in every situation, and there may be other subtleties that I'm missing.

You definitely cannot eyeball a non-trivial schema and figure that out in your head.

What we did at Riverbed was make one JSON document per API. This document was not a schema (which is one reason why I'm not proposing that Hyper-Schema adopt Riverbed's exact solution). The schemas were collected under a top-level types keyword. There was also a resources keyword, and the resources were listed under that. links were only directly defined on a resource for the "self" link and links sharing "self"'s URI template, or if they pointed outside of the set of resources described by our APIs (handwaving a bit here- omitting subtleties).

All other links were instead defined as relations, which explicitly named other resources from this or another API's resources section. To figure out the actual follow-able links, you went to the resource identified by the relation and looked at what links it defined. The relation definition mapped the source resource's properties into the destination resource's self URI template using vars.

This made the set of resources and the relationships among them explicit.

I could see just saying that "self" links define resources, and if you want to document the set of resources, just go through the schemas and find all of the "self" links. But the weird "figure out relations by figuring out if the URI templates resolve to the same URIs" part is where Hyper-Schema really falls down. The teams just totally balked at that.

targetSchema was mentioned a few times. If I have my way, this keyword would be removed entirely.

I'm almost 100% with you on this. As mentioned above, for connections between resources within the APIs defined by schemas, I prefer explicitly documenting the relationship, and then using the links on the relationship target. That completely removes the need for targetSchema for all of those cases (targetSchema for "self" was always a $ref back to the resource... because "self").

There are situations where you need to connect to something outside of JSON Schema. targetSchema and similar non-authoritative things are useful then. But they should otherwise be discouraged, or better yet made unnecessary.

While I like using the RFC 5988 terms as much as possible, we should not feel obligated to use parts of that that are better handled by other aspects of JSON Hyper-Schema.

from json-schema-spec.

@awwright

Regarding $ref in LDO, can you file a new issue that includes some examples and use cases?

#75

Please let me know if you'd like any additional details or examples, happy to provide.

from json-schema-spec.

@slurmulon Could you provide an example schema that is defining that link with the ambiguous {uuid}? I feel like I can almost see the issue and explain how I would approach it, but I'm not quite sure. To me, the rules for how to find the thing to use to fill in the template are all part of the schema, although a developer can choose to fill them in with external data.

Some variables may require external data, for instance if you want to link directly from a root resource to a specific element in the collection- the developer needs to supply the search term or id directly as there is no instance in which to find it.

In the approach we used:

1. URI template variable names must be unique
2. No complex structure was handled anywhere within the template names, it was entirely offloaded to vars and Relative JSON Pointer (this is why I view Relative JSON Pointer as essential).

And I guess I'll put an example here instead of offlist as it might be helpful for more people. This is adapted from a published API schema but I stripped out descriptions and irrelevant properties to focus it on the definitions of the resources, links, and relations. I also "fixed" things that were from earlier in the project and didn't match how we later sorted stuff out.

The links keyword is NOT a JSON Hyper-schema LDO. It is an object in which the keys are the LDO's rel value and the relevant properties are path (replaces href using the extended templating syntax, either a single string template or a template + vars object), method, request, and response (all pretty obvious- request is always the request body, while the URI parameters are handled by vars).

The relations keyword is also an object in which the keys are rel values. The properties are resource (a URI reference to the target resource declaration) and optionally vars, which does the same thing as it does with link paths, except where vars for links maps the instance's properties of the schema declaring the link into the link's own path template variables (or declares schemas for externally supplied value validation), the vars for relations maps the instance properties from the schema declaring the relation (the source side of the relation) into the URI template for the self link of the target resource in the relation.

Note that one relation is declared from within the "networks" array items, not on the entire "networks" schema.

I was going to explain more, but probably better to stop here and just let folks ask questions.

The example is in YAML, because everyone refused to write JSON by hand- somewhere there's a translated-to-JSON version but I can't find it right now and refuse to translate it by hand myself. Most JSON-heavy projects I've been involved with have it set up so humans write YAML and the JSON is produced as a build step.

types:
    some_type: {}
    etc_pretend_all_the_types_referenced_are_actually_here: {}
resources:
    networks:
        type: array
        items:
            allOf:
              - $ref: '#/resources/network'
            relations:
                full:
                    resource: '#/resources/network'
        links:
            self:
                path:
                    template: '/networks{?name,parent_id,virtual,is_default}'
                    vars:
                        name: {$ref: '#/types/unrestricted_name'}
                        parent_id: { $ref: '#/types/identifier'}
                        virtual: {type: boolean}
                        is_default: {type: boolean}
            create:
                method: POST
                request: {$ref: '#/resources/network'}
                response: {$ref: '#/resources/network'}
            set:
                method: PUT
                request: {$ref: '#/resources/networks'}
            get:
                method: GET
                response: {$ref: '#/resources/networks'}
    network:
        type: object
        properties:
            id: { $ref: '#/types/identifier' }
            name: { $ref: '#/types/unrestricted_name' }
            parent_id:
                anyOf:
                    - $ref: '#/types/identifier'
                    - type: "null"
                readOnly: true
        required: [name]
        links:
            self: {path: '/networks/items/{id}'}
            get:
                method: GET
                response: {$ref: '#/resources/network'}
            set:
                method: PUT
                request: {$ref: '#/resources/network'}
                response: {$ref: '#/resources/network'}
            delete:
                method: DELETE
        relations:
            uplinks:
                resource: '#/resources/uplinks'
                vars: {network_id: '0/id'}
            parent:
                resource: '#/resources/network'
                vars: {id: '0/parent_id'}
            child:
                resource: '#/resources/networks'
                vars: {parent_id: '0/id'}
    uplinks:
        # Just showing the self link to show how the network resource's "uplinks" relation maps to it.
        links:
            self:
                path:
                    template: "/uplinks{?name,network_id,site_id,parent_id,virtual,is_default,is_ps_capable}"
                    vars:
                        name: {$ref: '#/types/unrestricted_name'}
                        network_id: {$ref: '#/types/identifier'}
                        site_id: {$ref: '#/types/identifier'}
                        parent_id: {$ref: '#/types/identifier'}
                        virtual: {type: boolean}
                        is_default: {type: boolean}
                        is_ps_capable: {type: boolean}

from json-schema-spec.

One thing I forgot is that in later API definitions we decided that you should always be able to fulfill the self link from an instance, so for things like collections we included all search terms used in a meta section of the instance. It looked something like this:

resources:
    networks:
        type: object
        properties:
            elements:
                type: array
                items:
                    allOf:
                      - $ref: '#/resources/network'
                    relations:
                        full:
                            resource: '#/resources/network'
            filters:
                type: object
                properties:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: {$ref: '#/types/identifier'}
                    virtual: {type: boolean}
                    is_default: {type: boolean}
        links:
            self:
                path:
                    template: '/networks/{?name,parent_id,virtual,is_default}'
                    vars:
                        name: '0/filters/name'
                        parent_id: '0/filters/parent_id'
                        virtual: '0/filters/virtual'
                        is_default: '0/filters/is_default'

Pagination was handled the same way as filtering- URI template parameters of the query string variety, with any pagination values included in the instance under a pagination property (or something like that).

from json-schema-spec.

@handrews Going back to an earlier post, you bring up three points: Implicitly defined resources, Repetition in link definitions, and Can't describe both URI parameters and a message body.

(1) I don't fully understand the point with "Implicitly defined resources". A resource is anything that can be given a URI. So... pretty much anything. A diary entry, an HTML document, a company, a car. A resource isn't "defined" when it is linked to, a link defines a relationship to an existing resource.

A list of resources can be expressed with the "collection" pattern. There's loads of standards that perform this task, like Atom and Collection+JSON.

(2) I'm not sure what the issue is with "Repetition in link definitions". Links contain a subject (anchor) as well as an object (target). If I have a Link: </awwright>; rel=author then I'm asserting this resource has an author; not merely that "there exists an author".

(3) The fact that HTML forms can either append data to the URI query part, or pass data in the request-body, depending on the method, is an unfortunate anachronism. URIs don't actually define what the query-part has to look like, the typical key=value&... format is one defined by HTML. I don't think JSON Schema should be adopting it.

Finally, why does your schema have separate get/set/delete "links"? I'm still not entirely sure what's wrong with an array of links (i.e. LDOs or link templates).

from json-schema-spec.

@awwright : I'll answer these out of order :-)

get/set/delete "links": I almost left those out, but I wanted to show where the HTTP methods actually happen. There is nothing inherently wrong with the array of links, and I've said repeatedly that I don't intend to propose adopting Riverbed's approach wholesale. While I do prefer an object where the keys are rels, just because it is so much easier to work with in code, I am not at all attached to this particular links approach, and I can live with either a list or an object.

(3) I don't know what relevance HTML forms has to my point. The problem is that Hyper-Schema mandates that the schema parameter be applied to URL template parameters in the key=value form, which is just wrong. In part, because the query string need not be key=value (I'm always vastly amused that a near-universal usage is not defined by any standard anywhere, it just exists by existing). The other reason that it's a problem is that it implies that since GET query parameters are handled by schema, and schema is used for other things for other verbs, that there is no way to specify query parameters for other verbs. This is wrong.

The correct approach is to observe that URI Templates cover every possibility, including both traditional name=value style (either starting from ? or adding more with &) or any other style. Including excellent rules for mapping objects and lists into query strings in several different flexible ways. You can also just directly map a value into the query string and not use name=value at all.

So all that needs to be done is to provide a mechanism for mapping information from anywhere in the instance into the flat URI template variable namespace. vars does this elegantly and flexibly, and even handles the situation where the value cannot come from the instance.

(1) This is way, way, way, waaaaaaaay far off from what I was talking about. I wasn't talking about a list of resources like a collection. For now let me just try to phrase it this way: If I point you at one or more JSON Hyper-Schema documents defining a large API with, say, 100 resources in it, how do you, as a human, spot those 100 resources? How do you, as a human, determine which of those resources relate to each other? Take a look at my 4-step algorithm in response to @jdesrosiers for how I think about this.

(2) This isn't even in the same universe as the point I was trying to make :-) I'm going to have to go off and think about how to present this, as I obviously totally failed to articulate anything useful at all here.

from json-schema-spec.

@awwright one of the reasons for the separate get/set/delete links came from the Hyper-Schema document given an example of a link rel of "edit" which would use a method of PUT (the use of "set" rather than "edit" in our actual documents is unimportant).

from json-schema-spec.

@handrews wrt HTML forms, I bring it up because it appears the functionality was copied from HTML, even though it's a poor design and not really suitable for hypermedia in general.

Forms in HTML serve the purpose of both URI generators and remote execution. If you want to tell a person how to jump to page n in a paginated list, you use an HTML form with method="GET". On the other hand, if you want to tell a person how to create a new resource in a collection, you use an HTML form with method="POST". They're two totally different things, and they probably should have been entirely different HTML tags.

I think JSON Schema is sort of making the same mistake. There's no need to specify "method".

It seems you're being mislead by "edit", which is a link relation defined for use in Atom. If you want to change a document, you PATCH or PUT to the URI of the document you want to edit. There's no special link relation involved to edit things (or to get things, or to patch things). (There can be an "edit-form" link relation, which shows an HTML form to edit the current resource.)

from json-schema-spec.

@awwright I do think there's a place for "method" as not all REST APIs use HTTP, not all link relations indicate a method (HTTP or otherwise) and many things can be done multiple ways.

I need hyper-schema to indicate whether updating this resource is done with PUT or PATCH, and if it is PATCH then what media type should be used for the request? application/patch+json, application/merge-patch+json, or something else? I need to be told whether the response from an update is expected to include an updated representation, no body, or something else. Can I create with PUT? Or do I need to create indirectly through a collection or other resource with POST? I can probably go on.

As for the exact put/set/delete stuff in the example, I honestly don't remember how we came up it, or who exactly came up with it, and in any event I don't care about that part of the approach in the slightest. No one is advocating for using that format for links, so don't worry about it. It was just there to show how our system worked, not to advocate for replacing Hyper-Schema with that system (which, as proud as I am of that work, I would not advise anyway).

from json-schema-spec.

Updating a resource is something that can be done with either PUT or PATCH
depending on what the client thinks is most suitable. The server and the
link relation can advertise its support or nonsupport for a method, but
it's the server that should allow multiple ways to do things, it need not
lay them all out in the link or schema.

There's numerous standards like Allow, Accept, Accept-Patch, and so on, to
define the capabilities of the server.

Can you post some examples of what you think works better, please? I do
like the object of links like how HAL does, I forgot to add that to the
feature list.

On Oct 8, 2016 6:44 PM, "Henry Andrews" [email protected] wrote:

@awwright https://github.com/awwright I do think there's a place for
"method" as not all REST APIs use HTTP, not all link relations indicate a
method (HTTP or otherwise) and many things can be done multiple ways.

I need hyper-schema to indicate whether updating this resource is done
with PUT or PATCH, and if it is PATCH then what media type should be used
for the request? application/patch+json, application/merge-patch+json, or
something else? I need to be told whether the response from an update is
expected to include an updated representation, no body, or something else.
Can I create with PUT? Or do I need to create indirectly through a
collection or other resource with POST? I can probably go on.

As for the exact put/set/delete stuff in the example, I honestly don't
remember how we came up it, or who exactly came up with it, and in any
event I don't care about that part of the approach in the slightest. No one
is advocating for using that format for links, so don't worry about it. It
was just there to show how our system worked, not to advocate for replacing
Hyper-Schema with that system (which, as proud as I am of that work, I
would not advise anyway).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#48 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAatDeFvXfQCoYHA4Mw_eA2MDoSfKxFCks5qyEcLgaJpZM4J9x5n
.

from json-schema-spec.

I'm not concerned about advertising which methods are supported, I'm concerned with two things:

1. JSON Hyper-schema should provide an abstraction layer between API users and HTTP

A client should not be deciding what HTTP method is suitable. A client shouldn't be messing with HTTP methods at all- it should be reading link descriptions and doing what they say. The use of HTTP is determined by the scheme of the href value, and should not just be assumed anyway.

The possibility of non-HTTP links shows up in several places already- wording such as "In an HTTP environment, this might be..." in the section on method, and the entirety of section 5.4 "JSON Schema and other protocols" in the core spec: "JSON Schema does not define any semantics for the client-server interface for any other protocols than HTTP. These semantics are application dependent, or subject to agreement between the parties involved in the use of JSON Schema for their own needs."

Even when using HTTP, I really don't want to offload any discovery to OPTIONS because of this line from RFC 7231: "Responses to the OPTIONS method are not cacheable." JSON Schemas are indefinitely cacheable and (as we have discussed elsewhere), often pre-loaded in a client.

So if I'm building an API with JSON documents that does not run over HTTP, I may or may not have alternative standards to figure out what method(s) to use, assuming "method" is a translatable concept. I have definitely put in email link URIs to give a non-HTTP example (although method is not relevant to those).

1. Defining the resource's capabilities doesn't tell me how to use them

Yes, HTTP (assuming we're using HTTP) attaches semantics to some methods. But some of those semantics leave a lot of wiggle room. For something like PATCH, how should I use Accept-Patch? By making a non-cacheable OPTIONS call before every patch, or by ignoring the HTTP RFC and just doing it once?

And more importantly, how do I line up non-standard links that need to use POST with POST? If I don't recognize the relation name, do I do an OPTIONS on the URI and see if it allows POST and if so, just try that?

How do I figure out what to put in the body. The schema field? What about if I want to advertise validation specifics for PUT or PATCH bodies or responses? Is this where i use multiple links with the same URI? But the how do I figure out which is which?

OPTIONS is fundamentally broken which is why it's so poorly supported and rarely used (at least that I've encountered so far). And there are many things that OPTIONS cannot do, which is part of why we need Hyper-schema in the first place.

It's not burdensome for JSON Hyper-Schema to handle this stuff. The schema is the server's way of advertising what it supports, beyond the basics of HTTP. REST APIs are applications built on top of protocols and content types, whether that's HTTP and JSON or something else. Hyper-Schema should facilitate defining those applications, and not just assume that plain old HTTP handles everything that is needed.

I agree with you that Hyper-schema as it stands fundamentally muddles up the separate concepts of connecting two resources with a link and defining the operations available on the resource and how to execute them. But I believe that both are equally important for JSON Schema to specify.

from json-schema-spec.

@awwright OK here is my attempt to explain the duplication thing. This is also an indictment of how Hyper-schema manages HTTP methods. I very much think they should be removed from the specification of relationships between resources. But I also think it needs to be possible to specify them.

So if we instead changed method to methods and made it an object mapping methods to request and response schemas (and maybe some other stuff), that would be the minimal change needed to solve the duplication problem. It would not solve the explicit identification of all resources problem, but I'm going to have to spend more time coming up with an example for that.

Of course I could opt not to duplicate all of the target's methods for every relation, but that's also weird, and sometimes forces performing an unnecessary GET.

Anyway, here's the above example reworked into Hyper-schema (with extended templating).

Note: Please pretend the non-standard rels are proper URIs. It just made it really unreadable to make up full URIs for them.

definitions:
    networks:
        type: object
        properties:
            elements:
                type: array
                items:
                    $ref: '#/definitions/network'
            filters:
                type: object
                properties:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: {$ref: '#/types/identifier'}
                    virtual: {type: boolean}
                    is_default: {type: boolean}
        links:
          - rel: self
            href:
                template: '/networks/{?name,parent_id,virtual,is_default}'
                vars:
                    name: '0/filters/name'
                    parent_id: '0/filters/parent_id'
                    virtual: '0/filters/virtual'
                    is_default: '0/filters/is_default'
            method: GET
            targetSchema: {$ref: '#definitions/networks'}
          - rel: self
            method: PUT
            schema: {$ref: '#/definitions/networks'}
            targetSchema: false
    network:
        type: object
        properties:
            id: { $ref: '#/types/identifier' }
            name: { $ref: '#/types/unrestricted_name' }
            parent_id:
                anyOf:
                    - $ref: '#/types/identifier'
                    - type: "null"
                readOnly: true
        required: [name]
        links:
          - rel: self
            href: '/networks/items/{id}'
            method: GET
            targetSchema: {$ref: '#definitions/network'}
          - rel: self
            method: PUT
            schema: {$ref: '#/definitions/network'}
            targetSchema: {$ref: '#definitions/network'}
          - rel: self
            method: DELETE
            schema: false
            targetSchema: false

          - rel: create
            href: '/networks'
            method: POST
            schema: {$ref: '#/definitions/network'}
            targetSchema: {$ref: '#/definitions/network'}

          - rel: instances
            href:
                template: '/networks/{?name,parent_id,virtual,is_default}'
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: { $ref: '#/types/identifier'}
                    virtual: {type: boolean}
                    is_default: {type: boolean}
            method: GET
            targetSchema: {$ref: '#/definitions/networks'}
          - rel: instances
            href:
                template: '/networks/{?name,parent_id,virtual,is_default}'
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: { $ref: '#/types/identifier'}
                    virtual: {type: boolean}
                    is_default: {type: boolean}
            method: PUT
            schema: {$ref: '#/definitions/networks'}
            targetSchema: false

          - rel: parent
            href: '/netwoks/items/{parent_id}'
            method: GET
            targetSchema: {$ref: '#/definitions/network'}
          - rel: parent
            href: '/netwoks/items/{parent_id}'
            method: PUT
            schema: {$ref: '#/definitions/network'}
            targetSchema: {$ref: '#/definitions/network'}
          - rel: parent
            href: '/netwoks/items/{parent_id}'
            method: DELETE
            schema: false
            targetSchema: false

          - rel: children
            href: 
                template: '/networks/{?name,parent_id,virtual,is_default}'
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: '0/parent_id'
                    virtual: {type: boolean}
                    is_default: {type: boolean}
            method: GET
            targetSchema: {$ref: '#/definitions/networks'}
          - rel: children
            href: 
                template: '/networks/{?name,parent_id,virtual,is_default}'
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: '0/parent_id'
                    virtual: {type: boolean}
                    is_default: {type: boolean}
            method: PUT
            schema: {$ref: '#/definitions/networks'}
            targetSchema: false
          - rel: children
            href: 
                template: '/networks/{?name,parent_id,virtual,is_default}'
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    parent_id: '0/parent_id'
                    virtual: {type: boolean}
                    is_default: {type: boolean}
            method: DELETE
            schema: false
            targetSchema: false

          - rel: uplinks
            href:
                template: "/uplinks{?name,network_id,site_id,parent_id,virtual,is_default,is_ps_capable}"
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    network_id: {$ref: '#/types/identifier'}
                    site_id: {$ref: '#/types/identifier'}
                    parent_id: {$ref: '#/types/identifier'}
                    virtual: {type: boolean}
                    is_default: {type: boolean}
                    is_ps_capable: {type: boolean}
            method: GET
            targetSchema: {$ref: '#/definitions/uplinks'}
          - rel: uplinks
            href:
                template: "/uplinks{?name,network_id,site_id,parent_id,virtual,is_default,is_ps_capable}"
                vars:
                    name: {$ref: '#/types/unrestricted_name'}
                    network_id: {$ref: '#/types/identifier'}
                    site_id: {$ref: '#/types/identifier'}
                    parent_id: {$ref: '#/types/identifier'}
                    virtual: {type: boolean}
                    is_default: {type: boolean}
                    is_ps_capable: {type: boolean}
            method: PUT
            schema: {$ref: '#/definitions/uplinks'}

            child:
                resource: '#/definitions/networks'
                vars: {parent_id: '0/id'}
    uplinks:
        # Just showing the GET self link to show how the
        # network resource's "uplinks" relation maps to it.
        links:
          - rel: self
            href:
                template: "/uplinks{?name,network_id,site_id,parent_id,virtual,is_default,is_ps_capable}"
                vars:
                    name: '0/filters/name'
                    network_id: '0/filters/network_id'
                    site_id: '0/filters/site_id'
                    parent_id: '0/filters/parent_id'
                    virtual: '0/filters/virtual'
                    is_default: '0/filters/is_default'
                    is_ps_capable: '0/filters/is_ps_capable'

from json-schema-spec.

@handrews The problems you're describing aren't unique to JSON Schema, and for the most part, this all has known solutions.

The link itself is able to describe the features a server supports (including methods, media types, locales, and so on). But these attributes are advisory. In general, servers should be able to support a wide range of operations on resources, and clients opportunistically perform the operation they want to perform. If you want to edit a resource, you use PUT. If you want to make a small change, maybe PATCH is a better option for a client. HTTP does not need abstraction, HTTP is the abstraction by which you manipulate remote resources, and JSON Hyper-schema provides the means by which you discover new resources.

HTTP servers can in fact accept queries for and manipulate any resource with a URI. It's entirely legal, and an explicit feature, to send requests like:

GET urn:uuid:58db4c55-7611-41c5-9fff-aa3e58f57358 HTTP/1.1
GET ni:///sha-256;UyaQV-Ev4rdLoHyJJWCi11OHfrYv9E1aGQAlMO2X_-Q HTTP/1.1
PUT ftp://example.net/dir/file HTTP/1.1

Responses to these requests may be non-authoritative, but behavior for non-authoritative requests in HTTP is well defined.

When implementing the uniform interface, I don't think JSON Hyper-schema is doing a very good job of differentiating links from URI templates from remote execution.

• For linking, HTML has the <link/> and <a/> tags and the Link: header. You specify rel= and href=, and this specifies a relationship between two documents.
• For URI templates (specifying how to pull up a page n, or specifying how to pull up a page of search results for a query q), we use the <form method="get"/> tag. Together with input fields, the form tells a user how they can generate their own URIs to locate arbitrary resources. (while still leaving management of the namespace up to the server).
• Finally for remote execution, we use the <form method="post"/> tag. This is an entirely different concept than a link, it's telling us how to use an interface: How to format a document, submit it to a remote resource for evaluation, possibly modify the state of the server, then possibly return a new resource back to us for us to consume.

This remote execution ability is closer to the sort of concept you seem to be talking about: The server instructs the client exactly how to format a request-body (with enctype=), and which resource to upload it to (action=).

This pattern is not one followed in general. You don't need to tell a client how to GET a resource or how to PATCH a resource. It's something both client and server already understand because of the uniform interface.

There are link relationships that, when made, imply certain things about the link target. The "hub" link relation effectively means "Target is a resource that manages notification subscriptions for the current document. When POSTed to, it registers a notification endpoint..." This concept isn't terribly different than saying "if { <A> author <B> . }, then A must be a document and B must be an author".

Collections, likewise, are typically defined so that POSTing to them creates a new instance in the collection. No explicit link relation is needed, other than a link relation (or media type) that asserts the resource is a collection.

Take a look at http://amundsen.com/hypermedia/hfactor/ which documents the different kinds of links used in hypermedia.

The example I'm most interested in is what you think is the best way to define a Hyper-schema.

Pare down the example you gave to just the link relationships. You have three "self" and "parent" links that point to the same resource, there should only be one. Remove the "create" link relationship since it makes no sense to have an assertion { <A> create <B> . }. Replace "children" and "instances" with "item" if that's what you intend. Maybe organize the links into a key-value map like HAL does.

from json-schema-spec.

@awwright : The multiple links with the same rel are not something I think is good, they're me trying to illustrate the problems that the current approach to method and href forces on us.

Independent of whether you like my specific rel value choices, do you agree that JSON Hyper-schema, as it stands today in Draft 04, encourages this duplication? It's very important that we get clear on this, as if we do not agree on the starting point it is impossible to meaningfully discuss change. I feel like you are focusing so much on the solution of removing method that we're not getting clear on the current system's structural problems first.

@jdesrosiers @slurmulon how does all of this line up with your usage of Hyper-schema? Do you see the forced repetition problem that I believe is present? Or did you solve that another way or just not encountered a situation that would highlight this? I really want to understand if I am the only person who sees this concern with the system as it is now in Draft 04.

Let's get clear on Draft 04's problems before discussing solutions. Or critiquing my specific choices of rel: "instances", "create", "parent", and "children" all come directly from the hyper-schema specification either in its examples or defined by the spec (the example uses "up" instead of "parent" but it's close enough). I chose them in an attempt to avoid bikeshedding over the API design so we could focus on the structural problems of hyper-schema. I've already commented elsewhere that the spec ignoring the IANA "collection" and "item" relations is problematic, so getting lectured on using "item" instead of the examples that are in the spec is frustrating.

I'll address other stuff in separate comments. I really, really, really want to get agreement on the current problems with hyper-schema Draft 04 (and not API design problems with the example, especially not when I lifted the "bad designs" directly from the spec!) before proceeding.

from json-schema-spec.

@awwright : FWIW, you and I are in total agreement on this:

When implementing the uniform interface, I don't think JSON Hyper-schema is doing a very good job of differentiating links from URI templates from remote execution.

I also have thoughts on other parts of your last response. But I don't want to proceed with this discussion until I'm sure we're on the same page about what JSON Hyper-schema Draft 04 requires, and what problems it causes, without mixing in any discussion of solutions or the world outside of JSON Hyper-schema Draft 04. Then, once that's clear, we can proceed with discussing solutions.

from json-schema-spec.

@handrews, I don't think we are using the same definition of a "resource". To help illustrate, imagine an API for a bookstore. Among other things it has a definition of what the structure of a Book should be. The server maintains a list of Book instances. When I talk about resources, I'm referring to the instances of Books. I think you are referring to the structure of a Book. In OO terms, I'm referring to objects and you're referring to classes. You're concept of resource-oriented seems more like class-oriented (for lack of a better term). I think Hyper-Schema is so strongly resource-oriented that the classes can sometimes become a bit obscured. I'm ok with that. I think resource-oriented is the way to go.

@awwright, I see what you mean about LDO pulling triple duty, and I can see how splitting it up can have benefits. But, there is so much overlap between the three concepts that splitting them up could lead to unnecessary duplication as well. I'm not really sure.

Regarding the method keyword, are you saying that you don't think it is ever needed? Or, just shouldn't be allowed for links? In the third use of an LDO (forms), I don't see how it would make sense to not be able to specify a method.

Do you see the forced repetition problem that I believe is present?

@handrews, I don't really understand what you are getting at. But, no, I don't recall ever feeling like I was forced to repeat myself when creating Hyper-Schemas. The closest thing would be trying to describe a field that can only be written when the resource is created. The readOnly keyword is close, but a little too restrictive in this case. One solution has been to define a schema for creating a resource separate from the schema used for updating the resource even though they are mostly the same. This is the only case where I felt forced to repeat myself, but I'm pretty sure this is a very different issue than you are talking about.

from json-schema-spec.

@jdesrosiers : You have some good points with your resource and class analogy, but that's not actually how I'm looking at it. In fact, settling the terminology of "representation" (which is what schemas describe) vs "resource" (which is an abstraction generally** identified by a URI) was key for the success of the project at Riverbed. Conflating the abstraction of a resource with the concrete representation and calling them all resources created a muddled mess of confusion.

So your instances, to me, could either be resource instances (if we're discussing those books in the list abstractly) or instance representations (if we're talking about the JSON documents that get sent back and forth when I GET the list or elements of it).

To me, the schemas are classes, the representations are object instances (in the OO sense), and the resources are abstractions, the details of which are known only to the server.

So, schema, instance representation, resource instance, and resource type/class are the four key concepts here. They're not really interchangeable, although you can usually just say "resource" and it's usually clear enough from context whether it's a specific abstract book or the general abstract notion of books. On the other hand, we found that blurring the line between resource and representation was extremely confusing. The "manipulation of resources through representations" interface constraint of REST is a big clue that we should keep these concepts separate.

All resource instances of the same resource class should use the same schema for their representation (pluralize all of that if the resource class supports multiple representations). However, representations using the same schema may represent instances of different resource classes.

For instance, I might consider "publisher" and "distributor" to be two separate links from the "publication" resource class to the "company" resource class. In this view, they're both companies and the only thing that distinguishes them is the relationship to the publication.

However, I might consider publishers and distributors to be different concepts, and even though they are both represented in the same way initially as companies and therefore use the same schema for their instance representations, I might want to define them as separate resource classes to facilitate evolving them separately later on. Possibly even factoring out a company resource class later that publisher and distributor would both use.

Does that help or is it just more confusing? I'm glad you brought this up as I agree that it is critical for mutual understanding.

JSON Hyper-Schema is, unsurprisingly, schema-oriented :-) This makes it fairly representation-oriented as well, as the schemas are generally describing representations (either their structure or their hypermedia controls).

JSON Hyper-Schema is not, by this definition, resource-oriented. It is extremely hard to pick out what abstract resources are involved in a system at a glance. A schema with a "self" link is a representation of a class of resources. If "self" uses a plain URI, there is one instance of the resource, which has a representation matching the schema. If "self" uses a URI Template, there are many instances of the resource, each of which has a representation matching the schema.

But resource classes (and instances) are also identified by URIs or URI Templates in links other than "self" links. If the resolved URI matches another schema's self link, then those resources are the same and the representation is described by the schema containing the self link. But if the resolved URI only appears in a non-self link, then that is an additional resource, and the only hint we have for its representation is the non-authoritative targetSchema. If targetSchema is present at all.

Furthermore, figuring out whether a resolved URI Template in a non-self link matches some other self link's URI Template is non-trivial, which obscures the connections among resources. This is true whether we're talking about resource classes or resource instances.

If you still feel that hyper-schema is resource oriented, I would like to understand how you figure out the set of resource classes involved in a system, and the relationships among them, either at a glance as a human, or through a simple programmatic algorithm.

** There are also transient anonymous resources like error responses and many POST responses which have no self link or Content-Location header and therefore no identifying URI, but that's not important to the main point of this comment.

from json-schema-spec.

@jdesrosiers to connect this back to the example of the other format:

• The resources section gives a name to each resource class
• Each resource class's "self" link identifies the resource instances (one or many) of the class
• The schema given for each resource class is the schema for the primary representation of each resource instance
• The links (get/set/edit/delete/create) describe the circumstances under which the representation is used. GET 200 OK responses should always be the representation, as should PUT request bodies, and any response body that sets Content-Location to a URI matching the self link (that's all straight from RFC 7231). The request body of some POST requests (but not all) should a representation of the resource being created.
• The relations declare the relationships among resource classes, and instruct clients on how to map data from the representation of the source instance to the URI of the destination instance.

As @awwright noted earlier, we could deal with a lot of the when-to-use-the-represenation point by making assumptions based on HTTP. And in fact there was talk of adding a keyword like "operations": [get, set, delete] etc. that would indicate that those operations were supported in whatever way was standardized for the protocol in the URL scheme.

One reason we did specify them separately at first was that we also needed to describe APIs which abuse or under-utilize HTTP in various ways. Which, let's be honest, are probably more common than APIs which fully and correctly use HTTP, media types, etc. etc. I think it is important for hyper-schema to address all the things which are actually done in the wild, and not just the "correct" ideal. Although we should make the correct thing to do the easiest thing to do.

from json-schema-spec.

do you agree that JSON Hyper-schema, as it stands today in Draft 04, encourages this duplication?

I don't believe it does. "method" is an optional property in JSON Hyper-schema, and not present at all in most hypermedia technologies that have even fewer relation properties than JSON Hyper-schema provides.

Siren is the only other technology I'm aware of that lets you specify an arbitrary method, and only because it explicitly tries to copy much of HTML. I haven't seen an instance in Siren where people are listing otherwise-identical links for every method that the target resource supports.

Links are used for learning about new resources, so regardless of what JSON Hyper-schema defines, defining links multiple times doesn't achieve any additional effect for hypermedia purposes.

instead of the examples that are in the spec is frustrating.

Well, I'm asking for an example of what JSON Hyper-schema would look like ideally, and I'd like to see what you think that looks like.

The ultimate goal here is to assess which features are used most, and which features can be changed easily. Or if we need to make this assessment at all.

The examples you've provided suggests there's a lot of custom hacks and vocabulary, and not much attention would be paid to potential changes. Is that about right?

from json-schema-spec.

@awwright said:

The examples you've provided suggests there's a lot of custom hacks and vocabulary, and not much attention would be paid to potential changes. Is that about right?

No! Ugh. I regret putting those examples up and wish I could just rip this whole sub-thread out of this issue. The example with the resources section is from a company that I haven't worked for in nearly a year and a half, and I wasn't doing much with that project for the last few months I was there. And I have no current influence over it (I don't even know if they're continuing to expand it).

The reason I'm not putting an ideal version up is that you and I are not having the same conversation here. This is not an accusation, just an observation. I'm not entirely sure what to do about it.

I don't want to talk about solutions yet because there are clearly disagreements on the problems. So there's no point in proposing solutions. No one else seems to see the same problems, or in many cases have any idea what I'm talking about. Again, an observation and not an accusation- I am just apparently failing to convey what I need to convey.

I will think on it and try again later. Right now I'm a bit too dispirited.

I am hoping that you and/or @jdesrosiers will comment on my post about resource classes vs resource instances vs instance representations vs schemas. If we can come to an agreement on a theoretical basis (whether it is those four concepts or something a bit different), that would be a common point from which to start. Right now, we're talking past each other.

from json-schema-spec.

I think you guys need to have a Hangout chat, face to face for ten minutes to help adjust to each other's communication styles and expectations while ascertaining what the actual goals of the conversation are and next steps to solve them.

@awwright I still think a planning wiki, or even better, planning Google Docs, are a good idea. Context helps and annotated references can segment discussion and avoid ginormous comments like this thread is spawning.

Much of this is becoming either TL;DR or not clear enough to contextualise. I've found it's a common occurrence when you get a bunch of nerds like us in a room with no BA & PM to maintain focus 😆

from json-schema-spec.

@awwright @handrews @jdesrosiers wow, lots of moving parts here. I will definitely get back with some examples and comments, but I think @Anthropic is right - perhaps a different forum for conversation would help streamline things here.

Is there a Gitter set up for JSON Hyper-Schema? Not unable to find anything from a quick search, but I think that would help as it's real-time chat with persistent messages, like Slack.

from json-schema-spec.

@awwright I was considering splitting some threads out to separate issues, or starting threads on the google group (which doesn't seem to get much use). What would you prefer? Also happy to chat through some other system.

@Anthropic From digging through the old wiki, I'm not a big fan of using GitHub's wiki for planning- AFAIK there isn't really a good annotation and commenting system for it. I would be more in favor of Google Docs if we move to anything other than GitHub issues.

from json-schema-spec.

@handrews I definitely prefer Google Docs, being able to annotate directly on a sample schema is very helpful.

from json-schema-spec.

OK, @handrews, it sounds like we are using the same vocabulary after all. But, there does seem to be a difference between how we think Hyper-Schema fits into the REST architecture.

No one else seems to see the same problems, or in many cases have any idea what I'm talking about.

Yep. We're talking in circles. I think each of us has a different way of using Hyper-Schema and it's difficult for us to see other people's issues from our perspectives. Here is a great example.

If you still feel that hyper-schema is resource oriented, I would like to understand how you figure out the set of resource classes involved in a system

I wouldn't. The authoritative reference on what relations a resource has is the resource itself. A REST system has a reactive and dynamic quality to it. Documenting the system as a whole doesn't make sense to me. You learn about the system by browsing it. Each resource you get has it's own documentation.

I still think Hyper-Schema is resource-oriented. Using representations to interact with resources is how REST works and is what I mean by resource-oriented. I still think you are too concerned with a concept of classes which is a concept that doesn't exist in REST. Sorry I don't have time for a better explanation. There is one way that I think Hyper-Schema could be more resource-oriented. Currently, it only works with JSON and JSON-Schema. I don't think there is any reason why the hypermedia controls it describes couldn't apply to an XML instance described by an XML Schema. I think it should be possible to generalize Hyper-Schema to work with other Content-Types.

from json-schema-spec.

@jdesrosiers Hmm... maybe I shouldn't have used "classes". I really, truly, do not think of REST APIs in terms of OO. They are fundamentally different paradigms. I'll have to think more on how to convey this better. All I really mean by "class" here is that "all of these resource instances, as defined by the template self link, behave the same way." But REST behavior and OO behavior are not the same sort of thing, so "class" is misleading. I dislike "type" even more. Resources aren't types. Resource family? Open to suggestion here.

Also, you said:

I wouldn't [figure out the set of resource families in the system]. The authoritative reference on what relations a resource has is the resource itself. A REST system has a reactive and dynamic quality to it. Documenting the system as a whole doesn't make sense to me. You learn about the system by browsing it. Each resource you get has it's own documentation.

Yes, resources in a RESTful system inform clients what their relations are. They do this in ways that are media type and/or protocol specific, such as using link and anchor tags in HTML, or xlink in XML. When using HTTP, link headers can be used.

JSON does not have a mechanism for links. That is the point of JSON Hyper-Schema. The way a resource that is represented through JSON advertises its links is by providing a schema.

So I don't see the point of saying that a hyper-schema system that is built to describe links for JSON representation should not be used to describe links for a JSON representation. Am I missing something here? In this case, I don't mean whether the resource supports GET/PUT/whatever, I mean which other resources are linked to this resource. HTTP link headers are inadequate to convey all of the ways to fill out a complex URI template such as for a filtering system. Again, that's part of the point of Hyper-Schema.

If we're not relying on hyper-schema to describe hyperlinks, what are we doing with it?

from json-schema-spec.

@handrews For any problems you can identify with JSON Hyper-schema, I would strongly encourage you to file an issue describing the problem. Discussion here is fine too, but all the questions and responses I'm posing here are aimed at answering the question I opened this issue specifically to ask.

I'm afraid the definitions used so far are more complex than they need to be, let me put forward these definitions:

• A resource is anything that can uniquely be identified by a URI (so, pretty much anything). It can be anything from a JSON document, to a record in a database, to an employee, to a point in time. I think for JSON Schema, the HTTP definition is suitable https://tools.ietf.org/html/rfc7231#section-2, and some additional discussion is available from Wikipedia: https://en.wikipedia.org/wiki/Web_resource. Resources can be identified by any number of URIs.
• An information resource (IR) is any string of octets described by a media type. A png image and a JSON document would be information resources. Depending on the context, IRs might even be of indefinite length (for example, a live video stream).
• A document is another name for an IR. I additionally make the distinction that a document has a meaning given by its media type, and has a known length.
• A non-information resource (NIR) is any resource that's not an IR. For example, a blog post record in a relational database, or a company. Sometimes also known as an abstract resource.
• HTTP also calls an information resource a representation, and allows for the possibility that a URI can identify both an NIR and IR at the same URI: I see an "author" link relation (pointing to the author of the post), and when I dereference it, the server returns back an IR that has the same URI (e.g. a webpage about the author of the post). For most people on the Web, this isn't problematic. For the other 20% of us, there's a long-standing problem called httpRange-14. The definition of "resource" has changed slightly as we've had to iron out problems, but when REST says "Manipulating resources through representations", today we take this to mean that we can edit a JSON representation of a blog post, and the HTML representation will change too; or that I can use PUT to change the physical flipped state of a light switch.
• Older versions of HTTP also used the term variant, which I still use from time to time when discussing Content-Type negotiation and the Content-Location header. I don't anticipate this coming up in JSON Schema, though.
• I've refined the definition of "instance" and "schema" in the current HEAD: In the context of a specific validation or annotation operation, an instance is a JSON document interpreted according to the JSON Schema data model against a schema.
• A collection is a closed listing of resources. In context of belonging to a collection, the resource is called a member.
• A class or type is an open set of resources. In the context of having a type, a resource is called an instance. (A JSON Schema is similarly a particular kind of class, that has instances.)
• A profile is a particular kind of class that tells us how to interpret a document.

from json-schema-spec.

We do have an IRC channel, see http://json-schema.org/ for details.

from json-schema-spec.

@handrews, wow. you really misunderstood. Earlier I said,

it helps to think of the analogy to HTML as the JSON document + the Hyper-Schema.

That was what I meant when I talked about a resource. It's like how you don't think of a webpage as just HTML; it is also CSS, JavaScript, and images. It was perhaps a little sloppy use of language on my part, but I didn't want to confuse the concept with technicalities. That didn't work out as I hoped.

Even though a JSON document is a resource and a Hyper-Schema is a resource, in this context one is not complete without the other. I'll call this combination a "hyper-json" in order to stop using the term resource because it is apparently misleading. Hyper-jsons are the units that make up the APIs that I write. A hyper-json has no URI Templates because the Hyper-Schema URI Templates have all been evaluated to a concrete URI based on the JSON document it was paired with. These hyper-jsons are what make up the API. Hyper-Schemas are just templates and by themselves don't fully describe the API.

In order to get the kind of information you are talking about, you would have to crawl the API just like you would crawl a website.

from json-schema-spec.

I've made updates to JSON Hyper-schema in HEAD/master with the feedback here. I don't think it's reasonable to remove too much in the way of functionality, or add any functionality, and I figured out much of the current draft can be ironed out more smoothly:

• Reworked some language to be more formal-like
• updated the HTML reference to HTML5
• removed the special link relations only because nobody seems to use them
• removed the effect that "self" changes the URI base (this isn't the intended behavior for "self", and can be a different property altogether instead, like "base")
• clarified the language for "method"

The "method" section now looks like:

5.6.1. method

This property specifies that the client can construct a templated
query or non-idempotent request to a resource.

If "method" is "get", the link identifies how a user can compute the
URI of an arbritrary resource. For example, how compute a link to a
page of search results relating to the instance, for a user-selected
query term. Despite being named after GET, there is no constraint on the
method or protocol used to interact with the remote resource.

If "method" is "post", the link identifies how a user can construct a
JSON document to submit to the link target for evaluation.

from json-schema-spec.

@jdesrosiers

That didn't work out as I hoped.

LOL I think that has been true for each of us in this discussion a few times! Just in case it is not coming across in my writing, I am very appreciative of all of the discussion, opinions, and ideas, and am really enjoying how everyone is being so constructive. I hope my contributions come across that way even when careening off the rails :-)

In order to get the kind of information you are talking about, you would have to crawl the API just like you would crawl a website.

That's kind of my point- while if you want to find every concrete resource in the API, of course you have to crawl it. Hyper-schema will never tell you how many member resources are present in a collection resource, for instance.

But at the abstract level (what I was calling resource families) you don't need to walk over anything. Most HTTP APIs document these things be giving them a name, documenting the URI template, all of the methods and requests and responses, etc. This, to me, is a REST anti-pattern, because the exact URIs (or templates) are not something clients should get from documentation. They should be getting those through HATEOAS. Most APIs relegate far too much information to documentation.

But there is a vocabulary to a given REST API. Your client (human or otherwise) needs to know the names of the entities (resources and important fields) and the names of the relations among the entities. REST documentation should focus on this vocabulary. It is the part of the API that a human must comprehend in order to use the API, either by browsing or by writing a program. Some of the vocabulary can come from standards, but some will be domain (API) specific.

I am trying to figure out how I can read the hyper-schemas for an API and generate this sort of documentation instead of documenting a pile of URI templates and HTTP methods, requests, and responses. In the somewhat-regretted example that I put up, "network" and "uplink" are part of the vocabulary. There are no IANA-registered relations to explain how those two things fit together.

There must be some sort of documentation that tells client developers that this API deals with "network" and "uplink", and that makes it clear what those English words mean in the context of the API. The documentation should not get into the details of URI construction- that should be driven by what you call the hyper-json. But somehow I have to know that if I want to do stuff with networks and uplinks, this is the API that I use. And I have to know what relation to follow from the entry point resource to start, as there are no standard relations for these things. And I need to know what the representation of each of those concepts looks like (because once I know that, HTTP tells me a lot about where in various requests and responses I should use or expect to see that representation).

Perhaps another way to approach this discussion is to ask: What does your ideal REST API documentation look like? And what role do you see for hyper-schema in producing that documentation?

To recap, my ideal documentation names each resource family (or class or type or whatever we're calling it), describes its representation, and names the link relations that connect the various resource families to each other. Of course in practice, links connect specific resource instances to each other, but even though there are many instances of "network" and "uplink", we still just document what "network" and "uplink" mean once.

My ideal documentation does not publish URI templates or document the specifics of HTTP usage. It does need to include documentation of things like how to think about interactions that are not standardized, e.g. HTTP POST interactions other than those that create resources. It does need to document possible error resources (i.e. error response bodies as covered by the JSON Problem RFC- the codes are documented by HTTP), which is a critical topic of APIs which we have not even begun to discuss.

I've not yet seen any documentation of this sort (the docs generated from the Riverbed hyper-schema alternative still had way too much URI and HTTP information in them for my tastes, and not enough about error responses, but I lost that battle).

from json-schema-spec.

@awwright

This:

If "method" is "get", the link identifies how a user can compute the
URI of an arbritrary resource. For example, how compute a link to a
page of search results relating to the instance, for a user-selected
query term. Despite being named after GET, there is no constraint on the
method or protocol used to interact with the remote resource.

is a bad design smell if I've ever encountered one. If you just mean this as a clarification of the current state of things for Draft 05, I'm fine with it. But I think it's absolutely critical to separate methods (whether abstract or actual HTTP methods) from the definition of URI templates and the rules for filling them out from the representation. Any time your specification has to say "ignore this thing, it doesn't mean what it appears to me" then you have an unsustainable mess on your hands.

I'll attempt to reconcile the various concept definitions after getting some sleep. Are you getting your terminology such as IR/NIR from existing standards? I will happily adopt broadly-accepted terminology and definitions.

from json-schema-spec.

@handrews "Information Resource" and "Non-Information Resource" are formal terms used in Semantic Web technology, I don't believe there's a particular document that defines it all, but can be found in both formal and informal documents, e.g.:

• https://www.w3.org/TR/webarch/

• https://www.w3.org/TR/ldp-primer/

• https://www.w3.org/2001/tag/doc/httpRange-14/2007-05-31/HttpRange-14.html

  ---

You'll have to clarify what the design smell is, exactly.

I don't think the HTML design makes a lot of sense; but the way it's worded now not only preserves compatibility with existing implementations, but brings the definition into line with how HTML uses forms.

Even in HTML, method="get" and method="post" doesn't necessarily mean an HTTP operation using that method. Indeed, <form method="post" action="mailto:[email protected]"> ... is supposed to send an email, not make an HTTP request.

"get" and "post" may have been unfortunate choices of vocabulary for us human's sake, but at the end of the day they're just words. For all a computer cares, it could have been called "foo" and "bar".

If you're talking about discovering how to interact with a remote server, again, links are only for discovering the existence of new resources. Interaction with them is defined by the protocol you use to talk to a server, and their media type, if they have one.

I've laid most of the arguments for this design out, and I believe this is the current state of the art in hypermedia systems, but if you have a specific counterargument or believe this is wrong I can certainly consider it.

As for RESTfulness, it's certainly true that prose defining URI templates is not RESTful. In my understanding, servers can still provide clients with URI templates, however, and they can even cache these URI templates and use them within the expiry lifetime. The distinction is that for the latter case, the server always retains control of its namespace.

Leonard Richardson is the biggest pioneer of this concept, see https://www.infoq.com/presentations/REST-Hypermedia-Links-Forms for his wonderful explanation of this.

You probably knew this already, but it's worth stating explicitly.

from json-schema-spec.

There's one more thing I have to say here for the record just because I don't want to forget it:

Links on the Web follow the concept of progressive disclosure: The links that are shown are only the ones relevant to the resource they appear on. If a resource has been locked for editing, it does not make sense to provide a rel="edit-form" link.

We should examine how JSON Hyper-schema inhibits, if at all, progressive disclosure on the Web.

from json-schema-spec.

@awwright

I think we still need to be more clear on splitting the concept of discovering (and identifying) resources from describing any part of what operations can be done with the discovered resource.

First let me clearly align my terminology with yours (hopefully):

In the Riverbed hyper-schema alternative example, the resources section is (generally) how a client discovered new resources. The "self" link is how resources are identified (URI construction). The other entries in the links section describe operations on the resource (whether that is a good idea or not is still an open question). There is a notable problem which is that the keys in both resources and links are link relation names, which I think muddles the concepts considerably and has led to some of the major confusion.

Having explained all of that, let me definitively discard that terminology. I am on board with "links are only for discovering the existence of new resources." The "self" link also identifies resources using a given schema through specifying how to construct URIs. This is true both in Draft 04 hyper-schema and in the Riverbed format, so I think we're in agreement on that part. If you don't think that's a good way to think about the "self" link, please let me know of a better conceptual model.

So I'm fine with specifying links in that sense of "only for discovering of new resources." But I still have concerns around specifying operations. Currently this is handled by some combination of indicating a method of "get" or "post" in the LDO, and/or deferring to the protocol.

If links are only about finding resources, expressing any operation as part of the LDO at best distracts from that definition of links, and at worst is totally conceptually wrong.

I think we can all agree that HTML is not a good role model for more general hyperlinking. @awwright , you earlier said:

it appears the functionality was copied from HTML, even though it's a poor design and not really suitable for hypermedia in general.

Forms in HTML serve the purpose of both URI generators and remote execution. If you want to tell a person how to jump to page n in a paginated list, you use an HTML form with method="GET". On the other hand, if you want to tell a person how to create a new resource in a collection, you use an HTML form with method="POST". They're two totally different things, and they probably should have been entirely different HTML tags.

I think JSON Schema is sort of making the same mistake. There's no need to specify "method".

We're in agreement here (I think), which is why I am confused by the continued references to HTML to justify anything. Associating method with links is propagating HTML's problems into an area that has no need for it. Even in HTML, the <link /> element does NOT specify or imply a method. If we're going to follow HTML, this is what we should follow, not the anachronistic behaviors of <a></a> and <form></form>.

So again, if this wording is just to clarify the current behavior, that's fine- obviously we can't toss method and preserve compatibility. But if we're looking ahead, method should not be in the LDO at all. I can't tell from your reply whether you are advocating for continuing with this approach or just clarifying for the immediate next draft- if you could state which it is that would be help me with this a lot.

The "design smell" is the fact that specifying some particular method ("get") is supposed to be interpreted as not specifying an actual method, but just indicating that the link tells you how to get the URI of the related resource.

I'm not complaining about whether "get" indicates an HTTP GET or not. I'm fine with the values of method (wherever it lives in whatever form) being non-protocol-specific so that we can defer the specifics to the protocol (more on that later). It doesn't seriously bother me that the non-protocol-specific values happen to match two of HTTP's methods. That's probably not ideal, but it's waaaaay down on my list of concerns.

I'm complaining that a keyword called method is sometimes used for something other than defining a method. Basically, "post" does indicate that a method (specifically some protocol-determined non-idempotent method) is supported, but "get" does not say anything about method support. Let's just separate any description of methods/operations from the identification of related resources.

I'm going to address issues around deferring the specifics of interactions to the protocol in a separate comment, as they're orthogonal to the concerns in this comment.

from json-schema-spec.

@handrews Understood... I think. Let's open this on a new issue to re-examine for draft-06. Because no matter what this still needs work.

And let us know there if you think we've satisfactorily addressed the conserns well enough to release draft-05 that's supposed to be reverse-compatible.

from json-schema-spec.

@awwright will do, thanks. I will go over draft-05 and provide comments as soon as I finish responding to the last 24 hours' worth of activity :-)

from json-schema-spec.

Earlier I said:

I'm going to address issues around deferring the specifics of interactions to the protocol in a separate comment, as they're orthogonal to the concerns in this comment.

Nearly all of that is covered under issue #88 "problems" with relying on delegating to the protocol only really occur when the protocol is being abused. Which is very common with HTTP APIs.

from json-schema-spec.

@awwright

As for RESTfulness, it's certainly true that prose defining URI templates is not RESTful. In my understanding, servers can still provide clients with URI templates, however, and they can even cache these URI templates and use them within the expiry lifetime.

I definitely never intended to associate prose-defined URI templates with RESTfulness! Documenting a real REST API's URI templates is anti-RESTful. REST documentation should focus on the API's vocabulary rather than URI construction (or most other things that people tend to put in HTTP API documentation).

from json-schema-spec.

What's required to close this issue?

from json-schema-spec.

Pretty much all of my concerns have been addressed by splitting off #88 and #89, plus the other stuff that @awwright asked me to file separately which is in my increasingly large stack of partially written proposals. So I don't have any major outstanding concerns either.

from json-schema-spec.

@jdesrosiers Sorry to dig up an old issue, but regarding #48 (comment) (and having not re-read the full issue comments):

JSON Hyper-Schema is the only Level 3 option around.

I'm all for supporting JSON Schema (heck, I'm putting a lot of time into working on it!), but I recently read http://www.foxycart.com/blog/the-hypermedia-debate#.WMZyHBARDPc - Does that article not suggest 3 other standards which can attain RMM Level 3? Also, JSON-LD? I think I'm just starting to better understand Hyper Schema and HATEOAS, so trying to feel out what other people who have been doing this already feel.

from json-schema-spec.

@jdesrosiers thank you for the thorough response- great stuff!

I met Gregg Kellogg, one of the editors of the JSON-LD spec, at a W3C Web of Things conference recently, as the W3C WoT group is using a combination of JSON Schema and JSON-LD and invited each of us to discuss our respective (proposed) standards. There was a consensus that JSON-LD and JSON Schema Validation are solving different problems and that it would be good to at minimum work out some best practices for combining them. And probably do some work to make them easier to use together. I haven't followed up on this yet, but I will do so after Draft 06 is out the door.

I also evangelized Hyper-Schema a good bit, and there was some interest. Gregg brought up Hydra, but he's not much involved in it so there was no one there as a strong advocate for it. I've since read up on it, and I prefer our approach, even for Hyper-Schema + JSON-LD. Hydra is very focused on HTTP APIs and operations (exact HTTP requests/responses/methods/errors), rather than on defining links and letting the link relation and the protocol from the URL scheme determine possible operations.

Hyper-Schema does still needs a hints mechanism for things like HTTP Allow and Accept-Patch, but as of Draft 06 I think it's well-positioned to be usable in a system that discovers everything at runtime. To me, that is an essential difference: we (will eventually once again) treat HTTP usage specifics as an optimization, but allow the full runtime use of the protocol in a much more generic sense.

If your data is self-descriptive, you now have the possibility of automating things like interactions between systems where it usually requires a human to hard-code the connecting points. A form could be filled in automatically because the computer knows how the data it has matches up to the form data being requested. This can be achieved using JSON-LD in conjunction with a Level 2 hypemedia format.

This is the sort of thing I mean by clarifying how to use JSON-LD in conjunction with JSON Hyper-Schema :-)

Automating the connecting points is what #108 is all about. Some of that has now been implemented in Draft 06, and I might need to update and/or split what remains. But assuming we solve every use case (by whatever mechanism) then we will be fully capable of this sort of fully automated interactions.

Of course Internet/Web of Things people are much more interested in this than web app authors who have a couple of resources that are always used by programs encoding a specific simple workflow. IoT devices need a lot more flexibility and autonomy, so I suspect that community will drive a lot of "DMM Level 3" work.

from json-schema-spec.

Thanks for your replies @jdesrosiers @handrews

Just to calrify, because I'm a little unsure, but are you guys saying that you're including HyperSchema key words in your return json?

Unrelated to my question but in relation to your comments:
I was going to email you @handrews at some point soon to show you I'd found an example of someone using JSON Schema and JSON-LD, and I wasn't really sure what was going on. Interesting none the less. It's good we have open communication lines. I'm going to try and expand these to comapnies in the near future.

from json-schema-spec.

@handrews, it's great to hear that you are out there evangelizing Hyper-Schema. So few people seem to even realize that it exists. I never see it mentioned in discussions about Hypermedia APIs.

I'm very interested in better understanding how to use JSON-LD with JSON Hyper-Schema. I should admit that at this point DMM Level 3 is just theoretical to me. I have no practical experience with this yet and I don't know what it's limitations might be. I hope to be able to spend some time digging into this at some point.

@Relequestual, Hyper-Schema keywords are included in the JSON Schema, not the response body. In that way it is unique from other hypermedia formats. Check out this thread, https://groups.google.com/forum/#!topic/json-schema/40-rl7QKtiM. I think that should help explain how it works.

from json-schema-spec.

@jdesrosiers I should have a public blog post at my day job soonish that starts to explain how we're going to approach hypermedia. We'll also be building an open source generic hypermedia API client (initially in JavaScript, most likely).

I came close to DMM Level 3 in my work at Riverbed , which was Hyper-Schema-inspired rather than actual Hyper-Schema (long story, not relevant to future work). But we did actually use the sleepwalker.DataRep class that I linked to to automatically pull data from the current resource to construct the URI for the next resource. The only piece that was missing was the ability to pull data from the current resource to construct all or part of the request body for the next resource. Which is what I talk about in #108, and what I plan to do (hopefully with Hyper-Schema) for my current API work.

from json-schema-spec.

@Relequestual I include Hyper-Shema keyword in the response schemas, but not in the response instance data. That lets me use links from the response document, without the links appearing inline in the response.

from json-schema-spec.

@Relequestual actually it's more accurate that I define a resource's representation, including its links, under "definitions" and then always "$ref" that schema everywhere its needed, including requests and responses. This means that the links are always present wherever the representation is referenced.

So something like this, where I define a collection resource, an item resource, give them both "self" links and the appropriate side of the standard "collection" and "item" links, and use a custom link to connect the entry point to the collection:

{
  "definitions": {
    "identifier": {
      "description": "32-character hash identifier",
      "type": "string",
      "minLength": 32,
      "maxLength": 32,
      "pattern": "[0-9a-f]+"
    },
    "foo": {
      "$id": "#foo-item",
      "type": "object",
      "properties": {
        "id": {"$ref": "#/definitions/identifier"}
      },
      "links": [
        {
          "rel": "self",
          "href": "/foos/items/{id}",
          "hrefSchema": {
            "type": "object",
            "properties": {
              "id": {"$ref": "#/definitions/identifier"}
            }
          },
          "targetSchema": {"$ref": "#foo-item"}
        },
        {
          "rel": "collection",
          "href": "/foos",
          "targetSchema": {"$ref": "#foo-collection"},
          "schema": {"$ref": "#foo-item"}
        }
      ]
    },
    "foos": {
      "$id": "#foo-collection",
      "type": "array",
      "items": {
        "allOf": [
          {"$ref": "#foo-item"},
          {
            "links": [
              {
                "rel": "item",
                "href": "/foos/elements/{id}",
                "targetSchema": {"$ref": "#foo-item"}
              }
            ]
          }
        ]
      },
      "links": [
        {
          "rel": "self",
          "href": "/foos",
          "targetSchema": {"$ref": "#foo-collection"},
          "schema": {"$ref": "#foo-item"}
        }
      ]
    }
  },
  "link": [
    {
      "rel": "tag:example.com,2017:foos",
      "href": "/foos",
      "targetSchema": {"$ref": "#foo-collection"}
    }
  ]
}

from json-schema-spec.

Hyper-Schema keywords are included in the JSON Schema, not the response body.
@jdesrosiers

Thanks, that's what I though. I was pretty sure that was the case, but some of the discussion here confused me.

from json-schema-spec.

I include Hyper-Shema keyword in the response schemas, but not in the response instance data. That lets me use links from the response document, without the links appearing inline in the response.

@handrews

I'm not sure I totally understand the phrasing you've used there, but I'll paraphrase what I understand when in conjunction with your example and you can tell me if I'm getting it or not.

You include the collection and single link in both the collection and single definitions, so it's clear how to get from one to another. Cross referencing them in a sense. Right?

My only query from that schema is what you're trying to imply by having...

          "targetSchema": {"$ref": "#foo-collection"},
          "schema": {"$ref": "#foo-item"}

schema key word is about payload, right? Are you trying to imply that that URL is for both getting the collection of items, but also for posting / creating a new individual items? I would expect (and have done for my schemas to work with doca) define two seperate link elements for getting and posting, specifying the method.

Maybe this conversation should be taken sideways into email... Breaking my own rule of clogging up an issue... but it's a closed one so not so bad I guess.

from json-schema-spec.

@Relequestual @jdesrosiers I realized I did part of the example wrong. The "item" link should be from each item within the collection, not from the top level of the collection. That should make a lot more sense :-)

@Relequestual:
On the collection resource's self link, and the item resource's collection link, "targetSchema" is the collection. So if you GET on that link, you get the collection. If you PUT, you need to PUT the entire collection (unusual, but there are use cases for it and a collection is just another resource anyway).

On the item's self link, and each collection element's item link, "targetSchema" is the individual item resource. That's what you GET or PUT. Or PATCH, but that's a bit more complicated so I'm ignoring it for now. Let's get clear on the rest of it first.

On the item's collection link, and the collection's self link, "targetSchema" is the collection. Again, you GET and PUT that. But "schema" is the item.

An IANA "collection" link relation implies that a POST to the resource of an item representation will create a new item. While not mandated by the collection/item link relation RFC, this collection usage is basically the only design convention that's universally agreed-upon in all interpretations of "REST". An API should document that it does indeed support that, but this hyper-schema tells you everything you need to know to understand this.

The collection's "self" link has the same "schema" for the same reason, although at the "self" link relation does not determine anything about how POST is used. You need to notice that the "schema" schema has a "collection" relation that points back to yourself. Or notice that your "item" links points to the same thing as the "schema" schema. This is where things get a bit dodgy, and we need to put in more work. It needs to be fairly easy to recognize this sort of thing, but right now it's not entirely clear how that should work.

from json-schema-spec.

@handrews, it looks like we take somewhat different approaches. Your example looks like a service definition approach to me because you have everything defined in one big hyper-schema. What schema do you link to when someone requests a single foo? Do you duplicate #foo-item? Or, does it always refer back to the entry point schema somehow?

Below is how I would refactor your example to align with my approach. Each resource has a hyper-schema that applies to it. You might notice that I took out all uses of targetSchema. The reason I did that is because if it is a self link, it is understood that the targetSchema is the current schema. The explicit recursive reference is redundant.

1. A request to api.example.com/ is described by the following hyper-schema. It defines a link to /foos.

{
  "link": [
    {
      "rel": "tag:example.com,2017:foos",
      "href": "/foos"
    }
  ]
}

2. If the /foos link is followed, then a collection resource is returned and is described by the following hyper-schema. It defines a self link which can be used with GET, PUT, or DELETE where the targetSchema is understood to be this schema. It also defines a create link for adding a new foo. Because this schema references the foo-item schema, it has those links as well, including a link retrieve an individual foo.

{
  "$id": "http://api.example.com/schemas/foo-collection",
  "type": "array",
  "items": { "$ref": "/schemas/foo-item" },
  "links": [
    {
      "rel": "self",
      "href": "/foos",
    },
    {
      "rel": "create",
      "href": "/foos",
      "method": "post",
      "schema": { "$ref": "/schemas/foo-item" }
    }
  ]
}

3. If an individual foo is requested from a /foo/elements/{id}, it is described by the following hyper-schema. This includes the multipurpose self link as well as a link to the collection it belongs to. If the user follows the collection link, they go to part 2 where they have a collection and it's hyper-schema.

{
  "$id": "http://api.example.com/schemas/foo-item",
  "type": "object",
  "properties": {
    "id": {
      "allOf": [{ "$ref": "#/definitions/identifier" }],
      "readOnly": true
    },
    "foo": { "type": "string" }
  },
  "required": ["foo"],
  "definitions": {
    "identifier": {
      "description": "32-character hash identifier",
      "type": "string",
      "minLength": 32,
      "maxLength": 32,
      "pattern": "[0-9a-f]+"
    }
  },
  "links": [
    {
      "rel": "self",
      "href": "/foos/items/{id}"
    },
    {
      "rel": "collection",
      "href": "/foos"
    }
  ]
}

In case it wasn't clear, when I said "a resource is described by" I mean it has a Link header with rel describedby that can be dereferenced get a hyper-schema that can be applied to that representation (thus adding links to the response).

from json-schema-spec.

See, this is my problem. Just two people having different interpretations of what's expected by default. If it's not "well done" or unclear, then why leave it abigious when the spec allows you to be unabigious? I'll happily share my JSON Schema files when our API goes "live" to show you how I've done this, but as of right now I cannot share them.

from json-schema-spec.

@jdesrosiers There are a few things I just didn't convey well or did mistakenly:

• I just wrote a single schema for convenience. In a real system, the schemas will (almost certainly) be per-resource. There are some things to work out in terms of re-use vs per-schema versioning, but per-resource is the starting point.

• I don't even know why I put "targetSchema" on the self link. You are right that it is unnecessary and best left off, and I have written it that way elsewhere before.

• I'm with you on the "describedby" usage.

I'll comment again in a bit about where I still see differences. I need to think through the points I want to make a bit more first.

from json-schema-spec.

@Relequestual I want to separate two different sorts of concerns:

1. True hypermedia APIs are still not a well-understood thing, and designing them is challenging no matter your media types.
2. JSON Hyper-Schema does not yet provide all of the tools necessary to implement a comprehensive hypermedia design.

The first set of problems are not things that we can or should solve as part of JSON Hyper-Schema. An example relevant here is that IANA-registered link relations are often intentionally under-specified so that they can be used in a very broad set of circumstances.

The most significant difference between my schemas and @jdesrosiers's is in the choice of link relation type and the positioning of the links. I'm making an argument about what sort of assumptions you can make based on the presence of the IANA-registered "collection" and "item" link relations, even though the RFC that defines those relations does not require those assumptions, and therefore does not guarantee that they will always be correct.

But that has nothing to do with JSON Hyper-Schema. We could use XML and still find ourselves debating whether "collection" and "item" convey all of the necessary information.

What might be relevant is a question of whether JSON Hyper-Schema (or XML, or whatever) has or should have a mechanism of disambiguating standard link relation usage. Or describing custom link relation usage. Which I think is where @jdesrosiers's Relation Description Object proposal is coming from. But that's not specific to the choice of "collection" and "item" alone vs using a "create" relation.

Once you split the ambiguity up like this, it becomes clear that it's not a fundamental problem with JSON Hyper-Schema. Figuring out what assumptions can be made for a given link relation is independent of media type. Specifying assumptions is potentially part of the media type, but not specific to these link relations. And the differences in approach here come down to choice of an under-specified standard link relation vs a fully specified custom link relation (albeit a highly intuitive one).

from json-schema-spec.

@Relequestual, I think the problem is that there are no tutorials out there showing how it is supposed to work. This is especially true of draft-05 which was a significant change. My approach has been shaped primarily by one principle: It has to work in the Jsonary browser. If I can't navigate the entire API just by following links and filling in forms, then I'm doing something wrong.

Here is the example we have been discussing as a simple static site http://hyper-foo.s3-website-us-west-1.amazonaws.com/?url=example.json. None of the unsafe methods are supported, but those links are defined so you can see it work. Also, the schemas are modified a bit because it only supports draft-04.

Jsonary was created by the creator of JSON Hyper-Schema, so I see it as the reference implementation for the standard (draft-04 at least).

from json-schema-spec.

@handrews, thanks for the clarification. It sounds like we are on the same page (or at least very close) after all.

from json-schema-spec.

@jdesrosiers yay! :-)

As I alluded to in the last comment, I think our difference is mostly about choosing which link relations to use and where to put them, and about how much we can assume from the presence of a particular link relation. While that's somewhat orthogonal to JSON Hyper-Schema itself, let me go into a bit more detail because I think the "create" relation in this case in part motivated by a weakness in Hyper-Schema as it currently stands.

In general, I avoid relations like "create" that are actions, because the relation type name should describe the nature of the relation between the resource, rather than directly specifying a thing you can do with the resource.

The important part of this is in avoiding having to specify one LDO per operation on the same target resource. This is where I think Draft-04 went wrong with its use of "method" to explicitly state the exact HTTP method. The use of the HTML-ish "get" and "post" without exactly meaning "get" and "post" was (in Draft-00 through -03) and is (in Draft-05 and -06) a problem, but that was not the correct solution. BTW I recently updated my detailed analysis and history of "method" in Hyper-Schema to cover Draft-06 and better cover Draft-05.

For any link with an HTTP URI scheme (with or without the -S, also CoAP or anything else with an explicit mapping to HTTP), a link indicates that the following are possible:

• GET the resource, it should give its schema as the same one that's in "targetSchema", if any
• PUT the resource, using the "targetSchema" for the request
• DELETE the resource
• If "schema" is present, POST to the resource, using "schema" for the request

PATCH is more complicated as it should be expressed in terms of a patch media type, e.g. application/json-patch+json or application/merge-patch+json, and we don't have a good way to convey that right now.

We do not currently have an effective and well-designed way to express in Hyper-Schema how much of the above is actually supported, or if the resource behaves in any non-standard manners that violate these exceptions. A hit at the values of the "Allow" and "Accept-Patch" headers would fix a lot of that "Allow" would take a list of HTTP methods, to avoid duplication. If we had an "Accept-Patch" hint, or if clients did a HEAD to get the runtime "Accept-Patch", then the patch media type plus the "targetSchema" should tell us what we need to know, because patch media types tell us how to structure a patch for a given document.

This is why I do not think that Draft-04 or Jsonary are the right approaches. Both (along with nearly all documentation systems I have seen) are too operation focused. It's still leftover RPC. It should be resource-focused, and the possible operations are derived from the protocol indicated by the URI scheme. POST has no standard for describing its request format, so it is given explicitly. The only thing we are missing is a way to indicate design-time choices to disallow some methods/operations. Although...

Per RFC 5988, a link relation MAY specify that it implies certain operations (mapping to HTTP methods in this case) are or are not available. This is where I get the notion of assuming that a "collection" relation with a "schema" is telling me that I can create an item via that link. POST-to-create-in-collection is the only design pattern that is universally referenced by every guide to REST ever written. While the "collection" link relation RFC doesn't require it, I'd argue that anyone who violated that pattern in a RESTful API deserves whatever problems they get :-P

Note that not supporting creation and returning a 405 Not Allowed is fine. That's not violating the pattern, that's just opting not to implement it (a read-only collection). But specifying "schema" on a link with relation "collection" and having the associated POST do something else is just a horrible idea.

So where does that leave us? Well, I hope you can see why I am treating "collection" that way and not writing an explicit "create" relation, even if you don't find it compelling enough to adopt yourself. But mostly I agree with you that we lack tutorials and realty-tested best practices. For that, we just need to keep working on making this specification real, implemented, and running.

from json-schema-spec.

@handrews, I agree with almost all of what you say. Just two points where I disagree.

First, I don't share your concerns about the "create" relation. It looks like you are defining an action, but it can also be interpreted as defining a relationship to a resource that can be used to create something. Also, to be clear, this is the "create" relation that was defined in draft-04, not just some random description. However, I agree that "collection" could be used as well in this case. It may even be better. I'll have ponder that one a little longer.

But that brings me to the other point I disagree with: the idea that the "collection" link can be expressed as a single LDO. How schema is interpreted depends on the value of method. Without a value for method, the behavior of schema is undefined. You could do this ...

{
  "rel": "collection",
  "href": "/foos",
},
{
  "rel": "collection",
  "href": "/foos",
  "method": "post",
  "schema": { "$ref": "/schemas/foo-item" }
}

But, if you have schema without method,

{
  "rel": "collection",
  "href": "/foos",
  "schema": { "$ref": "/schemas/foo-item" }
}

there is no reason method should be interpreted as post. It could be interpreted as get which would make it a completely different link. Actually, I don't think it should be interpreted as anything and schema should be ignored.

from json-schema-spec.

@jdesrosiers:

This gets back to "method" being a confusing mess that needs removing. I should have included "method": "post" in all of my links in order to be unambiguously compliant with Draft 06 (I'll explain why that works in a bit). In Draft 06, the only thing that "method" does is determine whether "encType" and "schema" apply to the URL query string (as named params) or to the request body.

In Draft 06, the form-encoded URL query string and any and all other portions of the URI are more flexibly and comprehensively specified by "hrefSchema", which defines a schema for the URI Template variables. A template of "/foos{?offset,limit}" with an "hrefSchema" defining "offset" and "limit" properties in Draft 06 behaves identically to a "schema" defining "offset" and "limit" with a "method" of "get" in Draft 04.

Crucially, "hrefSchema" and "schema" can be used at the same time. This allows both defining a schema for URI query parameters (and other URI Template variables) and defining a schema for a request body.

So in Draft 06, there is never any reason to use "method": "get" except for backwards compatibility. The only truly useful value of "method" in Draft 06 is "post", so I tend to forget and leave "method" out entirely. Because it's confusing. You don't need it (because only one possible value is useful) and the values do not mean what people think they mean.

I have argued and continue to argue that having a keyword called "method" that takes two values that appear to be HTTP methods but are actually not HTTP methods is a horrible confusing approach that needs to be removed ASAP, and I will take up that cause again in Draft 07. It is also a problem for documentation or static inspection that JSON Hyper-Schema does not offer a mechanism for hinting at HTTP method allowance (hinting because the resource can always return 405 Not Allowed, possibly transiently due to application state). I also plan to revisit that in Draft 07.

But the documentation/hinting of HTTP method usage and the specification of URL query parameters vs request bodies are completely orthogonal problems, no matter what the HTML history is. HTML forms were designed to abstract the submission mechanisms away from human users, and have never evolved beyond their original limitations due to the rise of JavaScript. Web apps that want to do something other than the behaviors allowed by HTML forms get around it with JavaScript. That doesn't make any sense for JSON Hyper-Schema. It should be fully functional on its own, without requiring workarounds through code-on-demand.

Anyway, a Draft 06 "method": "post" link can be used with any and all HTTP methods, assuming you are respecting proper HTTP semantics (supporting abuse of HTTP semantics is a real-world practical issue and we'll need to look at that, too, but let's punt that for the moment):

• GET does not use "schema", as a GET payload has no defined semantics
• PUT does not use "schema", as Draft 06 explicitly notes that "targetSchema" should be used
• PATCH does not use "schema", and isn't currently well-supported by JSON Hyper-Schema
• DELETE does not use "schema", as a DELETE payload has no defined semantics
• POST uses "schema", as (unlike PATCH), there is no other way to describe its request structure

The request format for PATCH is determined by "targetSchema" plus the media type rules for the type(s) given in the "Accept-Patch" HTTP header. JSON Hyper-Schema doesn't provide a way to hint at or document "Accept-Patch", which is the same problem as the HTTP method hinting, which corresponds to the "Allow" HTTP header. We need to solve those problems, which is the right way to support PATCH (instead of overloading the use of "schema").

This is how linking works in most, if not all, hypermedia systems.

• RFC 5988 HTTP "Link" headers: No method. "rel" can imply restrictions on the method, but there is no direct hint.
• HTML <"link" /> element: No method.
• XML links: No method (that I can find- I might be wrong, there are five billion XML specs, it seems)
• JSON HAL: No method.
• Collection+JSON: Method not directly specified. Template can be used for PUT or POST

There are other JSON-based things that start describing methods, but they (and, I argue, Draft 04 of JSON Hyper-Schema) are confused about the difference between hypermedia (ensures that links can be recognized with enough protocol and link relation type information to allow a client to know how they could be used) and API specifications (describes the exact HTTP usage allowed for each instance of a link)

We need to figure out something about API specification functionality, because people want it. But it is not part of hypermedia, and we need to keep those concepts separate. This could be done through another JSON Schema vocabulary, or it could by done the way the validation and metadata keywords are handled in the JSON Schema Validation specification: each keyword is clearly designated to function as either a validation or a metadata keyword. No keyword conflates the two.

I think that ensuring that we separate those concerns, whether by vocabulary or just be documenting them as having different purposes, we will avoid a great deal of the confusion that is currently present. And perhaps that will also help us figure out a way to support APIs that abuse HTTP semantics without impacting either proper HTTP usage or documentation.

from json-schema-spec.

@jdesrosiers Also, I forgot "create" was in Draft 04 (it and "instances" were removed in Draft 05). "instances" is obviously redundant with "collection". "create"'s closest IANA-registered analogue "create-form", which is a better link name (relation, not action).

I think the "create-form" target resource needs to be the form rather than the resource to which the form is submitted, so that type of link would probably point to the collection's schema, because its "self" link's "schema" would be the form in question. I find just recognizing the "collection" link relation and applying the POST-to-create pattern more consistent, but I also see room for disagreement there.

from json-schema-spec.

It seems that we have different interpretations of how the schema keyword is supposed to be applied.

The way I see it, the schema keyword is a non-optional part of the link. You can't decide to ignore it because you are making a GET request and a GET isn't supposed to have a body. However, an HTTP client should recognize that it doesn't know how to construct a request for a link with a schema using anything other than POST and behave accordingly. (Our disagreement really had nothing to do with method, so I'll ignore that diatribe unless there is some part you specifically would like me to respond to. We are mostly on the same page.)

After some more thought about "create" vs "collection", I've decided that I very much prefer your pattern of using "collection". However, I can't get over the fact that it is undocumented behavior. It doesn't matter how intuitive or widely accepted the pattern is, we can't just assume a relation that was clearly and formally defined also has this other behavior. It would be nice if "collection" defined this behavior, but since it doesn't, the options are to create a new pair of "collection" / "item" -like relations, or use the existing relations and augment their behavior with something like "create".

I don't think that "create-form", as defined, can really fill the role that "create" played. A resource that is related by "create-form" is supposed to be a resource that contains a form. That doesn't really help. If we have a creation link on the foo-collection schema, we could have a "create-form" link on the foo-item schema that points to the collection. That IMHO is a proper use of "create-form". That doesn't mean I don't like your interpretation. I like the idea, I just think it differs from the intended semantics of "create-form". Of course we could always create a new relation with those semantics if we wanted. Maybe something like "create-schema".

from json-schema-spec.

@jdesrosiers

The way I see it, the schema keyword is a non-optional part of the link. You can't decide to ignore it because you are making a GET request and a GET isn't supposed to have a body.

Could you cite some motivation for this view in Draft 06? The JSON Schema specification is our specification. It means what we way it means. If there is something in Draft 06 that implies that "schema" must be used in this manner, that is a bug and we can fix it. It has been discussed multiple times in the process of writing Drafts 05 and 06 that a single LDO is intended for use with multiple HTTP methods, so the intent is clear to me. We just need to make sure that the intent is clear to the average reader of the spec.

However, I cannot find any such statements looking over Draft 06 right now. "targetSchema" explicitly states that it has different uses with HTTP GET vs PUT (capitalized so that they are clearly distinct from the explicitly lowercase "get" and "post" values of "method"). "schema" just doesn't say anything one way or the other. It is documented in the form of "if there is user input and method is X then user input is handled in way Y", which is not the same thing as "the presence of this keyword forces there to be user input." If you read it differently, please indicate exactly which parts imply required behavior so we can change them.

After some more thought about "create" vs "collection", I've decided that I very much prefer your pattern of using "collection". However, I can't get over the fact that it is undocumented behavior. It doesn't matter how intuitive or widely accepted the pattern is, we can't just assume a relation that was clearly and formally defined also has this other behavior.

This is exactly why I say that I see room for disagreement here. But this also comes up over and over with IANA-registered link relations. They are intentionally broadly defined, and will continue to be intentionally broadly defined. I don't know what the solution is (other than making up more "create"-ish relation types). RFC 5988 explicitly forbids treating combinations of relation types differently from relation types that are present separately, so "collection" + "create" should not be treated differently from "create" or "collection" individually.

One option is to document the usage of underspecified link relation types in a given API, but it's not clear to me how that fits with generic clients. It's not truly generic if you have to read non-standardized documentation.

This exact ambiguity is where I am struggling most with how to approach hypermedia.

from json-schema-spec.

I responded to some of this in #276, so I won't repeat that here.

"targetSchema" explicitly states that it has different uses with HTTP GET vs PUT

And it has no behavior with DELETE. The fact that the behavior of targetSchema is coupled to the HTTP methods just reinforces my belief that targetSchema is broken (since draft-05) and should not exist (since always). There is some grounds for your interpretation of schema based on the way targetSchema is defined, but I think it would be building upon a wart in the specification and not the direction we should be going.

RFC 5988 explicitly forbids treating combinations of relation types differently from relation types that are present separately, so "collection" + "create" should not be treated differently from "create" or "collection" individually.

Good point. I don't think that would be an issue for "create", but it would for the "create-schema" concept.

One option is to document the usage of underspecified link relation types in a given API, but it's not clear to me how that fits with generic clients. It's not truly generic if you have to read non-standardized documentation.

Agreed. In fact I've found very few of the IANA registered relations to be usable outside of the specific context they were designed for. Even "edit", which seems like it should be fairly generic, references constructs defined in Atom and therefore isn't reusable outside of Atom. Clearly something more is needed. RDO is the best way I've been able to come up with to address this problem.

from json-schema-spec.

The fact that the behavior of targetSchema is coupled to the HTTP methods...

I don't think it is. "targetSchema" hints at the schema for the representation of the target resource. It is not a response schema. HTTP semantics require that all of the following be a representation of the target resource, and therefore (assuming the hint is correct) be valid against "targetSchema":

• the response of a GET
• the request of a PUT
• the response to any request where the status is 200 and the "Content-Location" header is equal to the request URI

The semantics of PATCH define the request in terms of both the resource representation and the request media type, so there is an indirect relation between PATCH requests and "targetSchema"

Of course the representation of the target resource is irrelevant to a DELETE request, and with a POST request there is no guarantee of any relationship between the target resource representation and the request document. So "targetSchema" is irrelevant to those methods.

But all of this is defined by RFC 7231 (HTTP Semantics) and RFC 5789 (PATCH). It has nothing to do with JSON Hyper-Schema at all. "targetSchema" always means the same thing, but that thing applies differently to each HTTP method in a well-defined way.

RDO is the best way I've been able to come up with to address this problem.

I still find the RDO an interesting concept, I'm just not convinced that it belongs in the Hyper-Schema spec. I'm not 100% convinced it doesn't, either. Mostly I feel like I'm missing something about link relation types. Which is why I'm pestering mnot on mnot/I-D#211 :-)

from json-schema-spec.

After discussion elsewhere with @handrews and rereading this issue, the problem (for me at least) is the false assumption that I can use JSON Hyper Schema to document my non hypermedia compliant API.

I think what we need is a preface on how to determine if using JSON Hyper Schema is "right for my API", and how to identify (or define, I don't know) whay a hypermedia API looks like. Maybe explain a few different situations. It looks like draft-4 allowed the abuse of JSH to document non hypermeida APIs.

I'm still wrapping my head round what hypermeida IS and how it can be used.
I'm planning to collate a list of articles and documentation on JSON Schema in general, but also about hypermeida APIs and JSON Hyper Schema.

Given I gave a talk about JSON Schema and how to use it for a hypermedia API last week, and I'm still a bit confused about stuff, I think this should be our main address for draft-7.

from json-schema-spec.

@Relequestual, I don't think it's right to think about Hyper-Schema as documentation. I often refer to Hypermedia APIs as self documenting, but it's more about reducing the number of things that need documenting. It's kinda like if you referred to the method signatures in your classes as documentation. In a way they are, but they are also parseable, executable, and an integral part of the system.

from json-schema-spec.

Great thanks @handrews this will be super useful. I think as a first pass it should go on the wiki with a view to probably migrate it to the website once us non hypermedia understanding luddites feel the comparison and explaianition is sufficent to make sense =]

from json-schema-spec.

@Relequestual you might find my schemas useful...I'm currently adding/updating them daily...
https://github.com/eviratec/schema/tree/master/v1

Also see for examples of objects which are considered valid the schemas
https://github.com/eviratec/schema/tree/master/docs/v1

from json-schema-spec.

@handrews will you post an update here when done? I've always felt I didn't really understand the point of the hypermedia spec correctly coming from a different starting perspective.

from json-schema-spec.