http://zalando.github.io/restful-api-guidelines/data-formats/DataFormats.html#could-use-applicationspecific-content-types

The build of the Gitbook is broken due to an unintended upgrade to version 3.0.0

Looking at the PDF version generated after #12, I've found some things to improve.
I have not looked at the generation process, so I have no idea if they even can be improved at all.

• Each page's header has "Introduction", not only in the "Introduction" section.
• There are some spurious blank pages (3, 4, 7, 26) – one of them (page 4) is titled "Table of Contents", but is otherwise empty.
• The Section titles have same size as the guideline titles.
• There should be authorship and copyright mentioned somewhere (e.g. on a title page). I guess this ties in to #7. Maybe also the generation date (and/or commit date/commit ID from which it was generated?)
• At some points there is a page break right after a title (e.g. page 18).

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the Hypermedia section:

• rewording in the introductory section
• changed + formatted links
• added paragraph for the "Use URIs for custom link relations" rule
• added rule "Use Standards for Hypermedia Link Types" (this was later modified in #38, and will be addressed separately when we come to a decision).

Please review the diff and comment here.

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the Common Data Objects section:

• common address object:
  • wording change in introductory paragraph
  • extended description for salutation
  • change # to no
• common error return object: add "Hint: APIs may define custom problem types with extension properties, according to their specific needs.".
• change reasoning for the "no stack trace" rule.

Please review the diff and comment here.

In last API guild meetup we decided to update guidelines on links usage in payload and ids:

• all links should be present in _links field which can also be used multiple times in one payload
• favor simple IDs if in same namespace/system otherwise think about using URNs e.g. urn:warehouse:FG-AS-123-FD

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the Naming section:

• changed clarification for "Pluralize Resource names" rule
• reword rule " Consider Using Nested URLs When the Access Pattern Allows" → "Consider Using (Non-) Nested URLs" (with several changes in the text, too, and examples of non-nested URIs).
• added rules "Limit of Resources" and "Limit of Sub-Resource levels" → these were later removed in #23.
• some text changes to "Use conventional Query strings" rule

Please review the diff and comment here.

In the table accompanying the section Must: Use a well-defined subset of HAL you have a line for Collection+JSON with a row of Xs instead of checkmarks.

What does this table indicate and why is Cj getting the Xs? In particular what does the checkmark for the "docs" column represent? Is there something that you find missing on the "doc" front that might change this status?

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the Common Headers section.

(Later this was renamed to "Proprietary Headers" in #5)

• adding headers X-Frontend-Type, X-Device-Type, X-Device-OS, X-Uid with description
• extended description of X-Tenant-Id (this was later modified in #18)

Please review the diff and comment here.

There are some data fields that come up again and again and it's maybe worth writing them down -

• id: the identity of the object, often plugged into a url to be dereferenced.
• created_at: when the object was created as a date.
• updated_at: when the object was updated as a date
• type: the kind of thing this object is

The id field name is probably easy to get consensus around. The date fields are nearly as common but you could argue for different name conventions (eg created, created_date or created_timestamp). The type field is arguably not as ubiquitous as the other three, might not be needed as much given the use of OpenAPI, but it is common in public APIs.

Somewhat related is #62, which talks about the id field. Also as a datapoint the Google JSON Styleguide gives a lot of space over to reserved property names.

The README.md has two links pointing into Zalando internal infrastructure.

The current section doesn't contain a useful example. See HAL.

I last API guild meetup we decided to update HATEOAS rule and discuss hypermedia aspect more, in contrast links and relationships should be used anyway - regardless of HATEOAS.

As recently noted by @akauppi, the generated guideline docs have no link to this repository, so for people outside of Zalando's API guild (both in Zalando and from outside) it is not easy to find out how to provide feedback and/or improvement requests.

We should add such a link. Either something like the standard "fork me on Github" corner (which should be on each page – we would add this to the generator, I guess), or with a paragraph on the introduction page explaining how to propose changes. Or both.

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the HTTP section:

• added table "Redirection codes" in the "Use Meaningful HTTP Status codes" rule
• added some entries to the "Client side error codes" rule
• adding new table "Client side error codes" containing server side error codes.
• adding "Providing error documentation" rule (there are already #42 + #43 about this)
• rewording of the second paragraph in the gzip compression rule
• rewording of the first paragraph in the "Support filtering of resource fields" rule

Please review the diff and comment here.

I just noticed (looking for the April fools RFCs) that there is a new RFC 7807 defining a JSON error object, which seems to be based on the draft to which we are referring in the common data objects section.

That section should be adapted to refer to this RFC instead.

The RFC defines it as application/problem+json, while our guidelines are using application/x.problem+json. I guess we need some guideline about transitioning between both, because old clients might expect the content type with x (and old servers send it with x), while new clients would expect it without x (and new servers send it without x).

Hmm, now the README says "MIT License", while the LICENSE file says CC-BY (Creative commons Attribution 4.0). Sorry, I missed this in #19.

While these are largely equivalent (from my understanding), we should choose just one of them, or say that we license under both.

I usually love to use the browser search to quickly find what I'm looking for (e.g. when I have a keyword in mind and I know there's something in the guidelines about this).

The GitBook search doesn't seem so convenient. You still have to click through pages and don't have highlighted search results.

Is there a possibility to provide a single-page version of the book?

About managing the failed state.

In our distributed micro service world:

whenever a data point is created or manipulated a subsequent operation (on a different service) may fail and therefore invalidates the initial creation or manipulation of data.

Depending on the action, there should be the possibility to undo a previous action to get back into a clean state and allow a retry of the whole action later.

E.g. by using rollback operations for all non-idempotent calls (e.g. POST -> DELETE)

I would like to enforce the api guidelines so that there is always an idempotent operation in place or a do&undo pair.

An often discussed pattern for this transactional problem is e.g. the Saga Interaction Pattern. See https://www.youtube.com/watch?v=xDuwrtwYHu8&ab_channel=GOTOConferences

In Data formats there is {{ book.must }}. Should it be displayed as MUST?

In commit a68803b, a lot of changes were merged into the guideline without proper discussion. Those originated as change proposals in our internal Google doc, and had different states of approval at this point.

I was already surprised by several of those changes since then. I guess we should go systematically over all those changes and discuss whether they are good, and then possibly roll back the bad ones, or replace the text with better text.

This issue is a meta issue, we should create separate issues for individual points.

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the Pagination section:

• some wording changes to "must support pagination" and "Prefer Cursor-Based Pagination, Avoid Offset-Based Pagination" rules.

Please review the diff and comment here.

This issue contains a draft style guide for use with Zalando APIs akin to that provided by Google, for discussion. It could be a standlone document or a top level section of the existing guidelines (the latter would save having to work with two documents). There is something to be said for being able to think about media type and JSON content design independent of protocol concerns instead of having them all at the same level.

If we go ahead with this, the prs would probably be

• Create a top level section in the current guidelines and re-organize what's already in place to refactor the document. Existing guidelines will get moved around but won't have their specifications changed.
• Add in new guidelines as another pr.
• See if we want to break the JSON guidelines out to another document or leave them in place.

Note: sections below marked with an asterisk are extracted from the existing guidelines - the exact content below might be out of date with the exact guideline text in master, but that can be fixed up if this becomes a pr. In the meantime it's representative of what things would end up looking like.

JSON Guidelines

Table of Contents generated with DocToc

• Introduction
• Property Naming
  • Must: Property names must be snake_case (and never camelCase)*.
  • Must: Property names must be an ASCII subset
  • Should: Reserved JavaScript keywords should be avoided
  • Should: Array names should be pluralized
• Property Values
  • Must: Boolean property values must not be null
  • Should: Null property values should have their fields removed*
  • Should: Empty array values should not be null
  • Should: Enumerations should be represented as Strings
  • Should: Date property values should conform to RFC 3399*
  • May: Time durations and intervals may conform to ISO 8601
  • May: Standards may be used for Country, Language and Currency Codes*
• Common Objects
  • Should: Money should be represented using a common object*
  • Should: Addresses should be represented using a common object*
• Hypermedia
  • Should: HATEOAS should be used
  • Should: Standards should be used for Hypermedia Link Types
  • Should: Standards should be used for linked/embedded resources
  • May: URIs may be used for Custom Link Relations

Introduction

This guide provides recommendations for defining JSON data at Zalando. JSON here refers to RFC 7159, which updates RFC 4627 and the “application/json” media type. This guide clarifies some specific cases to allow Zalando JSON data to have an idiomatic form across teams and services and must be applied to JSON based APIs.

The keywords "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", and "MAY" in this document are to be interpreted as described in RFC 2119.

Property Naming

Must: Property names must be snake_case (and never camelCase)*.

No established industry standard exists, but many popular Internet companies prefer snake_case: e.g. GitHub, Stack Exchange, Twitter. Others, like Google and Amazon, use both - but not only camelCase. It’s essential to establish a consistent look and feel such that JSON looks as if it came from the same hand.

Must: Property names must be an ASCII subset

Property names are restricted to ascii encoded strings. The first character must be a letter, an underscore or a dollar sign, and subsequent characters can be a letter, an underscore, a dollar sign, or a number.

Should: Reserved JavaScript keywords should be avoided

Most API content is consumed by non-JavaScript clients today, but for security and sanity reasons, JavaScript (strictly ECMAScript) keywords are worth avoiding.

Should: Array names should be pluralized

To indicate they contain multiple values prefer to pluralize array names. This implies that object names in turn should be singular.

Property Values

Must: Boolean property values must not be null

Schema based JSON properties that are by design booleans must not be presented as nulls. A boolean is essentially a closed enumeration of two values, true and false. If the content has a meaningful null value, strongly prefer to replace the boolean with enumeration of named values or statuses - for example accepted_terms_and_conditions with true or false can be replaced with terms_and_conditions with values yes, no and unknown.

Should: Null property values should have their fields removed*

Swagger/OpenAPI, which is common use, doesn't support null field values (it does allow omitting that field completely if it is not marked as required). However that doesn't prevent clients and servers of sending (and accepting) those fields with null values.

Should: Empty array values should not be null

Empty array values can unambiguously be represented as the the empty list, [].

Should: Enumerations should be represented as Strings

Strings are a reasonable target for values that are by design enumerations.

Should: Date property values should conform to RFC 3399*

use the date and time formats defined by RFC 3339 — e.g. 2015-05-28T14:09:17+02:00 for a point in time (note that the OpenAPI format "date-time" corresponds to "date-time" in the RFC) and 2015-05-28 for a date (note that the OpenAPI format "date" corresponds to "full-date" in the RFC). Both are specific profiles, a subset of the international standard ISO 8601.

A zone offset may be used -- this is simply defined by the standards. However, we encourage to restrict to UTC (and go without offsets) when dates are passed and returned. From experience we know that zone offsets are not easy to understand and often not correctly handled. Note also, that zone offsets are different from local times that may include daylight saving time.

When it comes to storage, all dates should be consistently stored in UTC without zone offset. Localization should be done locally by the services that provide user interfaces, if required.

Sometimes it can seem data is naturally represented using numerical timestamps, but this can introduce interpretation issues with precision - for example whether to represent a timestamp as 1460062925, 1460062925000 or 1460062925.000. Date strings though more verbose and requiring more effort to parse, lack this ambiguity.

May: Time durations and intervals may conform to ISO 8601

Schema based JSON properties that are by design durations and intervals may be strings formatted as recommended by ISO 8601.

note: unsure if we need to spec the form of durations/intervals yet, or we should wait for implementation feedback/need. The duration guideline is taken from google's document.

May: Standards may be used for Country, Language and Currency Codes*

• ISO 3166-1-alpha2 country codes
• • (It's “GB”, not “UK”, even though “UK” has seen some use at Zalando)
• ISO 639-1 language code
• • BCP-47 (based on ISO 639-1) for language variants
• ISO 4217 currency codes

Common Objects

Should: Money should be represented using a common object*

Use the following common money structure:

Money:
  type: object
  properties:
    amount:
      type: number
      format: decimal
      example: 99.95
    currency:
      type: string
      format: iso-4217
      example: EUR
  required:
    - amount
    - currency

Make sure that you don’t convert the “amount” field to float / double types when implementing
this interface in a specific language or when doing calculations. Otherwise, you might lose
precision. Instead, use exact formats like
Java’s BigDecimal.
See Stack Overflow for more info.

Some JSON parsers (NodeJS’s, for example) convert numbers to floats by default. After discussing the
pros and cons,
we’ve decided on "decimal" as our amount format. It is not a standard OpenAPI format, but should
help us to avoid parsing numbers as float / doubles.

Should: Addresses should be represented using a common object*

Address structures play a role in different functional and use-case contexts, including country variances. The address structure below should be sufficient for most of our business-related use cases. Use it in your APIs — and compatible extend it if necessary for your API concerns:

address:
  description:
    a common address structure adequate for many use cases
  type: object
  properties:
    salutation:
      type: string
      description: |
        A salutation and/or title which may be used for personal contacts. Hint: not to be confused with the gender information that is stored per customer account
      example: Mr
    first_name:
      type: string
      description: given name(s) or first name(s) of a person; may also include middle names
      example: Hans Dieter
    last_name:
      type: string
      description: family name(s) or surname(s) of a person
      example: Mustermann
    business_name:
      type: string
      description: company name of the business organization
      example: Consulting Services GmbH
    street:
      type: string
      description: full street address including house number and street name
      example: Schönhauser Allee 103
    additional:
      type: string
      description: further details like suite, apt no, etc.
      example: 2. Hinterhof rechts
    city:
      type: string
      description: name of the city
      example: Berlin
    zip:
      type: string
      description: Zip code or postal code
      example: 14265
    country_code:
      type: string
      format: iso-3166-1-alpha-2
      example: DE
  required:
    * first_name
    * last_name
    * street
    * city
    * zip
    * country_code

Hypermedia

Should: HATEOAS should be used

This is a challenging topic. We don’t have specific recommendations yet; we expect them to emerge
from trial and error, and from the API Guild’s ongoing research. For now, we strongly encourage
teams to apply this principle and document results.

Notes on aspects to be analyzed:

• Security concerns with respect to passing OAuth2 tokens
• API discovery by client programmers vs. automated hypermedia driven workflows
• Additional client development work vs. reduced efforts in case of API and state handling changes

Additional resources: Wikipedia, The RESTful CookBook

Should: Standards should be used for Hypermedia Link Types

To represent hypermedia links in payload results, we should consider using a standard format like HAL, JSON API, JSON-LD with Hydra, or Siren. (Hint: Read Kevin Sookocheff’s post, for instance, to learn more about the hypermedia types.)

Should: Standards should be used for linked/embedded resources

For HTTP Link headers, consider using a format like HAL,JSON-LD with Hydra, or Siren. For a comparison of these hypermedia types, see
Kevin Sookocheff’s post.

May: URIs may be used for Custom Link Relations

Related or even embedded (sub-) resources should be linked via “meta link attributes” within the response payload; use here the following HAL compliant syntax:

    {
      “_links”: {
        “https://docs.team.zalan.do/rels/my-entity”: [{
          “href”: “https://service.team.zalan.do/my-entities/123”
        }]
      }
    }

favor simple IDs if in same namespace/system otherwise think about using URNs e.g. urn:warehouse:FG-AS-123-FD

relates to #54

From my point of view we should give a default implementation for generating the FlowId Header. Just a UUID in Java, for instance. What do you think?

Examples and the last paragraph of the data time format section are not consistent.

Are the time zones other than UTC allowed in payloads?

Please add LICENSE and MAINTAINERS files as described in our "How To Open Source" guide: https://github.com/zalando/zalando-howto-open-source

I did not find any guideline for rate limiting yet. Would be nice to include this. Widely used headers include:

X-Rate-Limit-Limit
X-Rate-Limit-Remaining
X-Rate-Limit-Reset
Retry-After

Related to #11

Currently we have two actual real world use cases where we use JSON Schema:

• Query API (Shop Search)
• Event Schema Registry (Nakadi)

We should discuss and agree on:

• When is it OK to use JSON Schema?
• What are some known shortcomings of OpenAPI that we know of (e.g. anyOf)?
• How should APIs be documented that use JSON Schema?
• How to embed JSON Schema in OpenAPI?
• How to describe purely event-driven APIs? Sample JSON files + Schema YAML files?

In "Could: Use Pagination Links Where Applicable", X-Total-Count is mentioned as an option for describing result count.

In JSON (which is the required media type), it's not necessary to ask clients to look in a header when a total count field can be added to the result entity, simplifying client access. The guidelines could be interpreted as saying when you serve pagination links, you are to use X-Total-Count in conjunction with paging links. It would be good to edit the example to include a total count in the data, and making X-Total-Count entirely optional. A final thought I guess is that we should consider avoiding custom headers to work with APIs unless they're strictly needed.

X-Total-Count seems to make most sense in these cases:

• The design style is to use the the Link header
• The response media type isn't able to carry the information
• The JSON response is an array instead of an object

The first two are not directly applicable (Link was removed a while back, we require JSON). The last one is (imho) something we shouldn't encourage because arrays don't afford extensibility the way object based collections/results can (eg you can't easily include pagination links with an array result in the example).

At least the enumeration in http://zalando.github.io/restful-api-guidelines/events/events.html#must-event-payloads-must-be-defined-in-accordance-with-the-overall-guidelines is out of sync with our current HATEOAS rules. We need to check this with other rules as well.

The guidelines currently say (in section "HTTP", rule "Must: Providing Error Documentation"):

The OpenAPI specification shall include definition of all error status codes and error descriptions
that possibly will be returned; they are part of the interface definition and provide important
information for service clients to handle exceptional situations and support troubleshooting.

I think we should instead encourage to not document common HTTP errors which have no special meaning for this API, but just the default HTTP meaning, and can happen always (like 500, 503, ...).

Clients should be prepared for unexpected response codes, and in case of doubt handle them like they would handle the corresponding x00 code. Adding new response codes (specially error responses) should be considered a compatible API evolution.

(This might also affect the compatibility section.)

To merge guideline changes we state in our contribution guideline that we need at least one approval of another committer. By zappr we can automate this process.

For API reviews and for discussions it'd be more convenient to have anchors for direct reference like http://zalando.github.io/restful-api-guidelines/common-data-objects/CommonDataObjects.html#CommonErrorObject

(This is a part of #41, see there for the general view.)

Commit a68803b changed those things in the Compatibility section:

• a small typo fix ✅
• completely reworded the itemized list in "Prefer compatible extensions" rule
• reword the "Avoid versioning" + "Use Media type versioning, not URI versioning" rules, leaving three rules "Avoid versioning", "Use media type versioning" and "Don't use URI versioning".
• reword the documentation version rule (without change of meaning, I think).

Please check the diff and comment here.

There for example.

Headline of each rule should express rule without further explanation.

I have seen attempts to implement idempotency with POST for simple CRUD services. Would be nice if the guidelines are more clear on this use case. An approach for handling ids should be suggested for services that would like to manage their own number ranges for ids (like another endpoint for id generation).

POST could also mention that returning 409 Conflict to realize idempotency is discouraged and a bad practice.

I think we should upgrade our used Gitbook version to version 3. It has breaking changes for us, we have to adapt some of our plugins.

By upgrading we can enjoy new features of Gitbook in future which can improve experience for readers of our guidelines.

(This is a part of #41, see there for an overview.)

Commit a68803b changed these things in the General Guidelines section:

• rewording of the Resources section
• space removal
• new section "Use domain specific resource names".

In the Design principles section those thing changed:

• link to internal document (Restful API practices) was removed, instead
• a generic sentence was added about the goals.

Please review the diff and comment here.

Several times during review I noted that API designers often think "Create = POST, Update = PUT".

The actual meaning is more like this:

• PUT means »please put the enclosed representation at the resource mentioned by the URL«, which will either create that resource (if it didn't exist before) or replace the resource currently there (if there was one). This should be used for creation when the client decides the URI where the resource will live.
  It has the additional benefit that it is idempotent, i.e. you can send the request twice without having two resources created (because the second one will simply update), and thus are resilient against connection failures.
• POST means (beyond other meanings like "fallback if no other method fits") »please add the enclosed representation as a new element of the collection resource identified by the URL«. The server will "invent" a new URI and put the new resource there. This should be used for creation when the server is deciding the resource identifier.

We should adapt the wording in the HTTP section to make this clearer.

We want to explore if it is possible to clearly separate between the parts where we are giving rules, and the parts where we are just giving guidance (which might be followed or not). (From the guild meeting.)

Right now we have two sections in the guidelines about limiting the number of resource/sub-resources to an arbitrary limit:

Should: Limit of Resources

To keep maintenance manageable, and to avoid violations of the “separation of concern” principle, an API should not expose more than 16 resources. If you exceed 16 resources, first check if you can split them into separate subdomains with distinct APIs.

Should: Limit of Sub-Resource Levels

There are main resources (with root url paths) and sub-resources (or “nested” resources with non-root urls paths). Use sub-resources if their life cycle is (loosely) coupled to the main resource, i.e. the main resource works as collection resource of the subresource entities. You should use <= 3 sub-resource (nesting) levels -- more levels increase API complexity and url path length. (Remember, some popular web browsers do not support URLs of more than 2000 characters)

I agree with limits, but I'm not sure whether we can have something as concrete as 16 in there. I'd consider 5 to be a lot of resources already. But there might be valid excuses for it.

Right now the guidelines state:

Should: Use Standards for Hypermedia Link Types

To represent hypermedia links in payload results, we should consider using a standard format like HAL, JSON API, JSON-LD with Hydra, or Siren. (Hint: Read Kevin Sookocheff’s post, for instance, to learn more about the hypermedia types.)

We probably should not allow all link standards in the long term, since we are going for a consistent API look and feel. To move that discussion forward, maybe we could start adding a comparison of different standards.

On Client Side Errors you define that 422 Unprocessable Entity should be use for semantic errors. Nevertheless you are limiting to POST, PUT, DELETE, PATCH HTTP verbs.

I understand the limitation because request entity is about Headers + body thus there is no sense for GET HTTP verb.

However GET can be requested with parameters (here query string parameter) that we could validate like:

/feed?url=XXX

Where url=XXX should be respect following validation:

1. Present, if not 400 Bad Request because is not semantic error
2. To be a valid URL (so not empty) (@URL)
3. To be a valid Feed URL

For 2. I really think is semantic error if user does not pass correct URL. Moreover there is also a edge case for post-validation (it may out-scope, 2. is the main goal of this issue), because it is not possible to determine validity without fetching it.

To conclude which status code should I return for 2. (and if you have advice for 3.)?

• 422 Unprocessable Entity seems to not appropriate because of Unprocessable Entity. There is no entity on GET request
• 400 Bad Request seems to be the least bad?
• 404 Not Found can be a great alternative and most RFC respectful but can deceive user.

Upstream discussion: zalando/problem-spring-web#49

With Open API, there's two basic ways to return a list of items

• arrays
• objects

Both are common use for APIs defined against these guidelines in Zalando.

I think it's worth recommending against array based responses and preferring object results. The main reason is that arrays design out extensibility, which is counter to the general principles. There's no way to extend an array with extra information or provide a home for common structures like pagination. This seems to lead to the definition of X-Headers to carry information (such as counts, but any kind of metadata really). Since Link headers have been dropped, APIs based on arrays have no real options for paging. Finally, there are odd bugs that can crop up when empty ends up meaning no more data in array responses, but also no data in the the current range, that are less likely to happen with explicit next/previous page declarations supportable by objects.

A knock-on consequence could be a need to firm up pagination models, and possibly defining an idiomatic structure for collections.

I think we should consider linking our recommended OpenAPI tools in the RESTful API guidelines:

• https://github.com/zalando/connexion
• https://github.com/zalando-stups/friboo
• https://github.com/zalando/play-swagger

(and others)

Obviously this would somewhat duplicate what is listed on the Swagger Open Source Integrations page, but IMHO a curated list in our guidelines has extra value as we already know which tools do not properly work and we specifically want API-first.

In section https://github.com/zalando/restful-api-guidelines/blob/master/data-formats/DataFormats.md we have defined the following rule: "Must: Use JSON as the Body Payload"

I think this is not 100% correct. We already required compliance to OpenAPI standard. Here payload is defined compliant to JSON -- especially via Schema Object http://swagger.io/specification/#schemaObject. It is a subset of JSON scheme. We also want to support more general JSON scheme structures -- Sean planed a corresponding rule proposal. @mostlymagic Could you do a corresponding update of this rule? Thx.