A model set of guidelines for RESTful APIs and Events, created by Zalando

  • By Zalando SE
  • Last update: Jan 2, 2023
  • Comments: 17

Developing Restful APIs: A Comprehensive Set of Guidelines by Zalando

Build status Latest published version: HTML, PDF, EPUB3

Purpose

Great RESTful APIs look like they were designed by a single team. This promotes API adoption, reduces friction, and enables clients to use them properly. To build APIs that meet this standard, and to answer many common questions encountered along the way of RESTful API development, the Zalando Tech team has created this comprehensive set of guidelines. We have shared it with you to inspire additional discussion and refinement within and among your teams, and contribute our learnings and suggestions to the tech community at large.

Usage

Feel free to use these guidelines as a guidance for your own development. Note that we encourage our own teams to use them in order to challenge their APIs. As such, you should consider this to be a living, evolving document. We will revise and update based on our learnings and experiences.

See BUILD documentation for technical details.

License

We have published these guidelines under the CC-BY (Creative commons Attribution 4.0) license. Please see LICENSE file.

Github

https://github.com/zalando/restful-api-guidelines

Comments(17)

  • 1

    Naming conventions for enum values

    There are no naming conventions that cover enum values.

    I've seen several different styles in our APIs so far:

    • MY_VALUE
    • my-value
    • my_value

    Does it make sense to try to standardize that to become more consistent?

  • 2

    Highlighting of sort key(s) in data change events

    The actual event schema definition does not explicitly state the sort key of data change events.

    Please create a guideline to highlight the sort key in the event schema definition taking into concern that this could be distributed across multiple properties.

    This is important to enable consumers to ensure the right event order during consumption.

    Thanks!

  • 3

    Deprecate HAL

    Deprecate HAL from the guidelines and replace it with local guidance designed to work with Open API.

    • since HAL has been subsetted to just links, all it effectively does is tell teams to create an object structure called _links. There's no real value in an underscore, we could at this point tell people to create an object called links and obtain the same benefits.
    • HAL is unfriendly to code generators [1] because it identifies the link using the rel as the key. This result in object structures that are awkward to deal with and drop relations that are unknown at compile time.

    The replacement guidance would (or could) be:

    • use links to enclose link structures in an array
    • define a Link object that contains a rel and href fields

    [1] For example using vanilla swagger you get an enclosing links object something like this

    public class PaginationLinks   {
      private PaginationLink prev = null;
      private PaginationLink next = null;
      ...
    }
    
    public class PaginationLink   {
      private String href = null;
      ...
    }
    

    Note the rels are mapped to field names in PaginationLinks. That means new rels will be dropped by clients meaning higher level code can't use them without recompilation. The Hypermedia guidance points out that keys are fields in its recommended object definitions:

    # keys are link relations
    

    But it doesn't point out new rels will be dropped on the floor. Regardless of your position on the guidelines aiming for Richardson L2 maturity, using keys this way for data that is intended to be extensible by the server isn't an optimal approach for Open API.

    An alternative model would be:

    links=[
     {
       "rel": "...",
       "href": "...",
     }
    ]
    

    That results in generated code that carries the rel and won't drop new rels sent by the server:

    public class MyCollection   {
      private List<Link> links = null;
      ...
    }
    
    public class Link   {
      private String rel = null;
      private String href = null;
      ...
    }
    

    It's closer in style to an a tag in html or rss/atom links. The downside is that finding a particular rel requires an iteration and is O(N). The argument is the number of rels in the loop isn't worth worrying about in general (we're talking single digit list sizes in the vast majority of cases, which is pagination or alternate references), and even it were, it's easy to do a one time pass into a map structure to provide an index.

  • 4

    Some tweaks for the API discovery section.

    Resulting from the review of a68803b in #41, some minor changes:

    • use example.com instead of myapi.com in the example URIs. (It was previously changed from myapp.myteam.zalan.do into myapi.com.)
    • mention that Twintip only understands JSON
    • the tool is still named "Swagger UI", not "OpenAPI GUI".
  • 5

    http 302 is missing in the list of [Success codes, Redirection codes, Client side error codes or Server side error codes]

    As per the use case, APIs are showing 302 Found when the try to create a resource which is already available and they give link to the available resource under the "Location" http header.

    Zally Violations details:
    The "should" tag has the following explanation "302 is not a well-understood response code".

    The rule 150 say is Below we list the most commonly used and best understood HTTP status codes, consistent with their semantic in the RFCs. APIs should only use these to prevent misconceptions that arise from less commonly used HTTP status codes. They are saying that 302 response code can be misconceptions with 301 and 303.

    Also RFC7231 says regarding 302 that the target resource resides temporarily under a different URI see more here: https://tools.ietf.org/html/rfc7231#section-6.4.3 .

    why this is not included in their Guideline, this sounds strange ? https://github.com/zalando/restful-api-guidelines

  • 6

    Avoids remote references in Event Type schema.

    This avoids remote references breaking the compatibility assurances available in event types, as the remote reference is not under direct control of the system managing the event types.

    For #237.

  • 7

    Add guidance on large numbers in JSON and interoperability

    JSON the specification doesn't provide a limit on the number production in its grammar but very long numbers or doubles with a long mantissa may lose precision when processed by common implementations (such as JavaScript).

    rfc7159 allow arbitrary long number productions in its grammar but says implementations may impose limitations without saying what they are. This tends to result in interoperability problems and disagreements on who's responsible for the issue.

  • 8

    AsyncAPI specification

    Hi! I just came across your API guidelines and found it awesome. And what I liked the most is that you also document your events using OpenAPI schemas. I just wanted to know if you're aware of the AsyncAPI specification. It seems like a good fit for your use case and its schemas are exactly the same as OpenAPI schemas.

  • 9

    398 quotation marks

    fix #398

    Two commits:

    1. Replace the »...« by "..."
    2. Replace the "..." by “...” everywhere

    Options:

    1. We keep the ASCII quotes "..."
      • Pros
        • Easier to type using any keymap
        • Probably what is already used when pasting text from other sources
      • Cons
        • Not typographically accurate
        • There is already a mix of "..." and “...” in this documentation
    2. We switch everything to “...“
      • Pros
        • Typographically accurate
        • Align the entire documentation to one quotation standard
      • Cons
        • Bigger diff (not really a problem)
        • Hard to maintain (people will probably have the reflex of using "..." in the future).

    I did it in 2 different commits, which will be useful if we go with the 1st option (rollback the 2nd).

    Let's discuss!

  • 10

    Should collections consume 2 resources in "Limit number of Resources [146]"?

    Currently the Zally implementation literally counts the number of path entries in the swagger file and ensures that this number is below some configured paths_count_limit (currently 8). This has the effect that any collection of things instantly burns up two of the allowed paths /things and /things/{id} - is that the intention of this guideline?

    In my team we were considering a variation where we just put a limit on the count of top-level resources i.e. /things, /things/{id}, /things/{id}/related-thing/ all count as a single top-level resource, where /extra/, /extra/stuff/ would count as a second top-level resource. It occurred to me that this might have been the original intention of the rule?

  • 11

    Migrating to Asciidoc

    This PR implements the migration from Markdown/GitBook to Asciidoc(tor) and includes:

    • Translation from Markdown to Asciidoc markup
    • New build and deployment scripts
    • Introduces RuleId concept and concrete rule ids (and tooling around them)
    • Support for the existing links (redirects to the current rule locations)
    • Adjusted documentation
    • Updated Maintainers list

    Please also have a look at the generated HTML.

    @tfrauenstein Feel free to award the rule ids [1,99] to the most prominent rules.

    Related to #231, #69, #10, #238. Closes #260.

  • 12

    Re-review the status codes

    With RFC 9110 (HTTP semantics), some status codes are now more clearly specified.

    We should look into whether to update our status code list.

    Candidates:

    • 307, 308, 422, 417
  • 13

    Warn about downstream failures of JSON `\u0000` encoding

    While \uxxxx are valid characters in a JSON string, they can create failures when leaving the JSON context, e.g. by writing to a database or piping it to through tools. While most tools may handle this gracefully, there is at least one known exception:

    • Postgres cannot handle \u0000 in strings in the jsonb type (because the null character is not allowed in text) (see datatype-json).

    Consequently, services that forwarding JSON content to sensitive tools must check their input and reject or sanitize characters not supported by their tooling.

    A good candidate for adding this warning would be rule #167, however, may be this would be not prominent enough and we should create a new rule: {MUST} sanitize JSON payloads from critical characters

  • 14

    Add rule copy link buttons

    Rule headings could contain a variety of small icons at the end of them for quick copy-pasting into documents, chats, and comments.

    I've thought about two specifically for now - the second one of which would be most handy for PRs and GitHub issues.

    1. 🔗 - Regular link

    This just copies the URL to clipboard and nothing else. One less action compared to now :)

    2. - Markdown link

    Copies a Markdown-formatted link to clipboard so you can very easily reference it in e.g. a GitHub issue. This would make it easy to produce meaningful links instead of raw URLs or manual editing.

    Some format ideas:

    1. [MUST use standard formats for date and time properties [169]](https://opensource.zalando.com/restful-api-guidelines/#169)
    2. [MUST use standard formats for date and time properties](https://opensource.zalando.com/restful-api-guidelines/#169)
    3. [REST [169] MUST use standard formats for date and time properties](https://opensource.zalando.com/restful-api-guidelines/#169)
    4. [RESTful #169](https://opensource.zalando.com/restful-api-guidelines/#169)
    5. [REST #169](https://opensource.zalando.com/restful-api-guidelines/#169)
    

    Markdown icon source: https://commons.wikimedia.org/wiki/File:Markdown-mark.svg

  • 15

    Unclear scope of recommendation against total count remark

    https://github.com/zalando/restful-api-guidelines/blob/4d7b9d0ffd2ff65bd0ad5252c367aa49beac4dae/chapters/pagination.adoc#should-use-pagination-links-where-applicable

    The "You should avoid providing a total count..." remark in that section can arguably be thought to apply to the whole concept of pagination instead of just the "SHOULD use pagination links where applicable" one where it is structurally located.

    Would it be possible to clarify the intent somehow? For example, by moving it to the "MUST support pagination" section it would be clearer that it applies to all of the chapter's alternative pagination approaches, or by amending the remark with a few words like "You should avoid providing a total count along with pagination links unless there is a clear need to do so." would make it clear that it applies to this particular section only.

  • 16

    Optimize the Event Guidelines for its usage with Nakadi

    The Event Guidelines partially are designed to be generic, i.e. agnostic with respect to the Event Registry and Event Bus used to exchange events (e.g. Nakadi or AWS SQS). On the other hand Nakadi-specific aspects are distributed all over the place and there is no clear differentiation between generic parts and Nakadi specific parts. This creates inefficiencies for guideline usage and maintenance for the main use case at Zalando with 98% events being exchanged via Nakadi, while it does not really cover event exchange via other technologies line AWS SQS. For instance, it creates redundancies (e.g. Event Type schema definition in guidelines and Nakadi API spec) that tend to evolve inconsistently, and rules that are not needed (e.g. MUST use semantic versioning) since they are automated by Nakadi.

    Proposal: Focus the event guidelines on Nakadi, eliminate guideline redundancies (e.g. rule 197, rule 246) with Nakadi doc and Nakadi API, inter link all documents clearly.

  • 17

    some formatting can't be converted to PDF

    Problem Pages 10 (durable references) and 44/45 (common method properties table) of the PDF show some pieces of text as plain HTML instead of formatted properly.

    Analysis

    When running the make all locally, I get some errors:

    ...
    docker run -v /home/pebermann/projektoj/restful-api-guidelines:/documents/ asciidoctor/docker-asciidoctor:latest asciidoctor \
      -D /documents/output index.adoc;
    docker run -v /home/pebermann/projektoj/restful-api-guidelines:/documents/ asciidoctor/docker-asciidoctor:latest asciidoctor-pdf \
      -D /documents/output index.adoc;
    asciidoctor: ERROR: failed to parse formatted text: ​<code><a href="https://infrastructure-api-repository.zalandoapis.com/" class="bare">https://infrastructure-api-repository.zalandoapis.com/</a></code> <b>–</b> used to refer to user defined, immutable API specification revisions published via the internal API repository.
    asciidoctor: ERROR: failed to parse formatted text: ​<code><a href="https://opensource.zalando.com/restful-api-guidelines/{model.yaml}" class="bare">https://opensource.zalando.com/​restful-api-guidelines/​{model.yaml}</a></code> <b>–</b> used to refer to guideline defined re-usable API fragments (see <code>{model.yaml}</code> files in <a href="https://github.com/zalando/restful-api-guidelines/tree/master/models">restful-api-guidelines/models</a> for details).
    asciidoctor: ERROR: failed to parse formatted text: <span class="should-keyword">&#x26a0;&#xFE0F;</span> No, but <a anchor="229"><span class="should-keyword"><strong>SHOULD</strong></span> consider to design <code>POST</code> and <code>PATCH</code> idempotent</a>
    asciidoctor: ERROR: failed to parse formatted text: <span class="should-keyword">&#x26a0;&#xFE0F;</span> May, but only if specific <a href="#post"><code>POST</code></a> endpoint is <a anchor="safe">safe</a>. <strong>Hint:</strong> not supported by most caches.
    asciidoctor: ERROR: failed to parse formatted text: <span class="should-keyword">&#x26a0;&#xFE0F;</span> No, but <a anchor="229"><span class="should-keyword"><strong>SHOULD</strong></span> consider to design <code>POST</code> and <code>PATCH</code> idempotent</a>
    asciidoctor: ERROR: failed to parse formatted text: <span class="should-keyword">&#x26a0;&#xFE0F;</span> No, but <a anchor="229"><span class="should-keyword"><strong>SHOULD</strong></span> consider to design <code>POST</code> and <code>PATCH</code> idempotent</a>
    asciidoctor: ERROR: failed to parse formatted text: <span class="should-keyword">&#x26a0;&#xFE0F;</span> May, but only if specific <a href="#post"><code>POST</code></a> endpoint is <a anchor="safe">safe</a>. <strong>Hint:</strong> not supported by most caches.
    asciidoctor: ERROR: failed to parse formatted text: <span class="should-keyword">&#x26a0;&#xFE0F;</span> No, but <a anchor="229"><span class="should-keyword"><strong>SHOULD</strong></span> consider to design <code>POST</code> and <code>PATCH</code> idempotent</a>
    mv -f output/index.pdf output/zalando-guidelines.pdf;
    ...
    

    It looks like the HTML→PDF conversion did break here somehow? The only thing I can see which those have in common is that some more or less complicated HTML is included via a {...} directive into the actual code here.