API tutorials beyond OpenAPI

Not all documentation is created equal. According to the popular classification, there are four document types: tutorials, how-to guides, technical references, and explanations.

OpenAPI, the de facto standard for documenting APIs, is a decent reference-style documentation (and client code generator, of course). But it can't serve as a good how-to or tutorial.

In this article, I will introduce a concise and readable way to write interactive tutorials and how-tos for any HTTP API (REST, RPC, or other style). And for that (surprise, surprise), we will rely on the HTTP protocol itself.

A crash course in HTTP messages

HTTP/1.x is a plain-text protocol that describes the communication between the client and the server. The client sends messages like this:

POST /anything/chat HTTP/1.1
host: httpbingo.org
content-type: application/json
user-agent: curl/7.87.0

{
    "message": "Hello!"
}

And receives messages like this in response:

HTTP/1.1 200 OK
date: Mon, 28 Aug 2023 07:51:49 GMT
content-type: application/json

{
    "message": "Hi!"
}

HTTP/2, the successor to HTTP/1.1, is a binary protocol. However, all tools (such as the browser devtools or curl) display HTTP/2 messages in plain text (just like HTTP/1.1), so we can safely ignore this fact for our purposes.

HTTP request consists of three main sections:

1. Request line:

POST /anything/chat HTTP/1.1

• The method (POST) defines the operation the client wants to perform.
• The path (/anything/chat) is the URL of the requested resource (without the protocol, domain and port).
• The version (HTTP/1.1) indicates the version of the HTTP protocol.

2. Request headers:

host: httpbingo.org
content-type: application/json
user-agent: curl/7.87.0

Each header is a key-value pair that tells the server some useful information about the request. In our case it's the hostname of the server (httpbingo.org), the type of the content (application/json) and the client's self-identification (user-agent).

3. Request body:

{
    "message": "Hello!"
}

The actual data that the client sends to the server.

The HTTP protocol is stateless, so any state must be contained within the request itself, either in the headers or in the body.

HTTP response also consists of three main sections:

1. Status line:

HTTP/1.1 200 OK

• The version (HTTP/1.1) indicates the version of the HTTP protocol.
• The status code (200) tells whether the request was successful or not, and why (there are many status codes for different situations).
• The status message is a human-readable description of the status code. HTTP/2 does not have it.

2. Response headers:

date: Mon, 28 Aug 2023 07:51:49 GMT
content-type: application/json

Similar to request headers, these provide useful information about the response to the client.

3. Response body:

{
    "message": "Hi!"
}

The actual data that the server sends to the client.

There is much more to the HTTP protocol, but this basic knowledge is enough to cover most of API use cases. So let's move on.

Using HTTP to document API usage

We are going to take an HTTP request:

POST /anything/chat HTTP/1.1
host: httpbingo.org
content-type: application/json
user-agent: curl/7.87.0

{
    "message": "Hello!"
}

And modify it just a little bit:

• include the full URL in the request line instead of the path;
• remove the protocol version.

POST http://httpbingo.org/anything/chat
content-type: application/json

{
    "message": "Hello!"
}

This format is perfect for API usage examples. It's concise and readable, yet formal enough to be executed programmatically (directly from the documentation, as we'll see shortly).

Writing an interactive API guide

Instead of telling you how to write an interactive API tutorial, I'm going to show you one. We'll use Gists API as an example. It's a compact and useful GitHub service for storing code snippets (called "gists").

Even if you are not a GitHub user, you still have access to the Gists API.

Reading gists

Let's take a look at the public gists of my pal Redowan (user rednafi). The response can be quite chatty, so we'll only select the 3 most recent (per_page = 3):

GET https://api.github.com/users/rednafi/gists?per_page=3
accept: application/json

A family of non-standard x-ratelimit headers tell us how GitHub limits our requests:

• There is a total number of x-ratelimit-limit requests available per hour.
• We've already used x-ratelimit-used requests.
• So there are x-ratelimit-remaining requests left.

We need to keep an eye on these to make sure we don't exceed the quota.

We can use a combination of page and per_page query parameters to select a slice of gists. For example, here are gists 10-15:

GET https://api.github.com/users/rednafi/gists?page=3&per_page=5
accept: application/json

Note that GitHub provides navigation links in the link header:

link:
    <https://api.github.com/user/30027932/gists?page=2&per_page=5>; rel="prev",
    <https://api.github.com/user/30027932/gists?page=4&per_page=5>; rel="next",
    <https://api.github.com/user/30027932/gists?page=7&per_page=5>; rel="last",
    <https://api.github.com/user/30027932/gists?page=1&per_page=5>; rel="first"

That's thoughtful of them!

Okay, now let's take a look at the specific gist with id 88242fd822603290255877e396664ba5 (this one is mine; let's not bother Redowan anymore):

GET https://api.github.com/gists/88242fd822603290255877e396664ba5
accept: application/json

We can see that there is a greet.py file written in the Python language with a certain content:

class Greeter:
    def __init__(self, greeting):
        self.greeting = greeting

    def greet(self, who):
        print(f"{self.greeting}, {who}!")

gr = Greeter("Hello")
gr.greet("world")

(yep, you can also create interactive Python examples!)

Interestingly, the gist has a history. It appears that every time you edit a gist, GitHub creates a new version, while also keeping previous versions.

Let's get the earliest revision, which has a version = 4c10d27cfb163d654745f1d72f2c7ce14225b83b (a bit long, I know):

GET https://api.github.com/gists/88242fd822603290255877e396664ba5/4c10d27cfb163d654745f1d72f2c7ce14225b83b
accept: application/json

The code in the gist was much simpler back then:

msg = "Hello, world!"
print(msg)

Modifying gists

Okay, so we know how to list gists for a user, how to get a specific gist, and even how to get a specific revision. Now let's create a new gist!

POST https://api.github.com/gists
content-type: application/json
accept: application/json

{
    "description": "Greetings in Markdown",
    "public": true,
    "files":{
        "README.md":{
            "content":"Hello, world!"
        }
    }
}

What's that? We have a 401 Unauthorized error. The response body explains: "requires authentication" and even provides a link to the documentation (oh, I just love GitHub APIs).

Understandably, GitHub does not allow anonymous users to create new gists. We have to authenticate with an API token.

If you want the following examples to work, enter your API token in the field below. You can create one with a 'gist' scope in the GitHub settings.

After you enter the token below, it will be stored locally in the browser and will not be sent anywhere (except to the GitHub API when you click the Run button).

Let's try again, this time with an authorization header.

Note the public parameter. The service supports "secret" gists (public = false), but it's the "security by obscurity" type of secrecy. Secret gists do not show up in the "GET gists" API method, but they are still accessible by id, even by anonymous users.

POST https://api.github.com/gists
content-type: application/json
accept: application/json
authorization: bearer {token}

{
    "description": "Greetings in Markdown",
    "public": true,
    "files":{
        "README.md":{
            "content":"Hello, world!"
        }
    }
}

HTTP/1.1 201 
cache-control: private, max-age=60, s-maxage=60
content-length: 3758
content-type: application/json; charset=utf-8
etag: "819f6b4f728843abcb50ad63da200a4c110245585b3eb1c0f59a5ebe86c8ecf5"
location: https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9
x-accepted-oauth-scopes: 
x-github-media-type: github.v3
x-github-request-id: E8B5:8EDA:511F73:51AC33:64EE0266
x-oauth-scopes: gist
x-ratelimit-limit: 5000
x-ratelimit-remaining: 4997
x-ratelimit-reset: 1693323114
x-ratelimit-resource: core
x-ratelimit-used: 3

{
  "url": "https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9",
  "forks_url": "https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9/forks",
  "commits_url": "https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9/commits",
  "id": "b17474320a629af38255c0a6efbc72b9",
  "node_id": "G_kwDOACz0htoAIGIxNzQ3NDMyMGE2MjlhZjM4MjU1YzBhNmVmYmM3MmI5",
  "git_pull_url": "https://gist.github.com/b17474320a629af38255c0a6efbc72b9.git",
  "git_push_url": "https://gist.github.com/b17474320a629af38255c0a6efbc72b9.git",
  "html_url": "https://gist.github.com/nalgeon/b17474320a629af38255c0a6efbc72b9",
  "files": {
    "README.md": {
      "filename": "README.md",
      "type": "text/markdown",
      "language": "Markdown",
      "raw_url": "https://gist.githubusercontent.com/nalgeon/b17474320a629af38255c0a6efbc72b9/raw/5dd01c177f5d7d1be5346a5bc18a569a7410c2ef/README.md",
      "size": 13,
      "truncated": false,
      "content": "Hello, world!"
    }
  },
  ...
}

HTTP status 201 Created means that a new gist has been created as a result of our request.

Okay, now we can update a gist using its id (don't forget to replace the {gist_id} in the request line with the actual id value):

PATCH https://api.github.com/gists/{gist_id}
content-type: application/json
accept: application/json
authorization: bearer {token}

{
    "description": "Greetings in Markdown",
    "public": true,
    "files":{
        "README.md":{
            "content":"¡Hola, mundo!"
        }
    }
}

HTTP/1.1 200 
cache-control: private, max-age=60, s-maxage=60
content-type: application/json; charset=utf-8
etag: W/"989eaec7cdb50ba6441e77ea2defba257b98a535f26c2ba6062f152ceffb2d77"
x-accepted-oauth-scopes: 
x-github-media-type: github.v3
x-github-request-id: E8B5:8EDA:5188AA:52163F:64EE027F
x-oauth-scopes: gist
x-ratelimit-limit: 100
x-ratelimit-remaining: 98
x-ratelimit-reset: 1693323129
x-ratelimit-resource: gist_update
x-ratelimit-used: 2

{
  "url": "https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9",
  "forks_url": "https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9/forks",
  "commits_url": "https://api.github.com/gists/b17474320a629af38255c0a6efbc72b9/commits",
  "id": "b17474320a629af38255c0a6efbc72b9",
  "node_id": "G_kwDOACz0htoAIGIxNzQ3NDMyMGE2MjlhZjM4MjU1YzBhNmVmYmM3MmI5",
  "git_pull_url": "https://gist.github.com/b17474320a629af38255c0a6efbc72b9.git",
  "git_push_url": "https://gist.github.com/b17474320a629af38255c0a6efbc72b9.git",
  "html_url": "https://gist.github.com/nalgeon/b17474320a629af38255c0a6efbc72b9",
  "files": {
    "README.md": {
      "filename": "README.md",
      "type": "text/markdown",
      "language": "Markdown",
      "raw_url": "https://gist.githubusercontent.com/nalgeon/b17474320a629af38255c0a6efbc72b9/raw/95975f3d0bac707ce4355dfc4a7955310d212fac/README.md",
      "size": 14,
      "truncated": false,
      "content": "¡Hola, mundo!"
    }
  },
  ...
}

It now greets us in Spanish 🇪🇸

Very good. Finally, let's delete a gist:

DELETE https://api.github.com/gists/{gist_id}
accept: application/json
authorization: bearer {token}

HTTP/1.1 204 
x-accepted-oauth-scopes: 
x-github-media-type: github.v3
x-github-request-id: E8B5:8EDA:51E584:5273CC:64EE027F
x-oauth-scopes: gist
x-ratelimit-limit: 5000
x-ratelimit-remaining: 4996
x-ratelimit-reset: 1693323114
x-ratelimit-resource: core
x-ratelimit-used: 4

HTTP status 204 No Content means we deleted the gist, so GitHub has nothing more to tell us about it. It's a little sad to see it go, but we can always make another one, right?

The Gists API has other useful features, but they are beyond the scope of this tutorial. Here are the functions we've covered:

• List user gists.
• Get a specific gist, or a specific revision of a gist.
• Create a new gist.
• Update an existing gist.
• Delete a gist.

Now try managing your gists! You can always use this article as a playground.

Implementation

To run the API examples as we did in the previous section, you'll need a bit of JavaScript that does the following:

1. Parses the HTTP request example.
2. Calls the API.
3. Displays the result.

Since we've limited ourselves to a small subset of HTTP request capabilities, parsing is fairly easy:

// parse parses the request specification.
function parse(text) {
    const lines = text.split("\n");
    let lineIdx = 0;

    // parse method and URL
    const methodUrl = lines[0].split(" ").filter((s) => s);
    const [method, url] =
        methodUrl.length >= 2 ? methodUrl : ["GET", methodUrl[0]];
    lineIdx += 1;

    // parse headers
    const headers = {};
    for (let i = lineIdx; i < lines.length; i++) {
        const line = lines[i].trim();
        if (line === "") {
            break;
        }
        const [headerName, headerValue] = line.split(":");
        headers[headerName.trim()] = headerValue.trim();
        lineIdx += 1;
    }

    // parse body
    const body = lines.slice(lineIdx + 1).join("\n");

    return { method, url, headers, body };
}

const spec = parse(`GET https://httpbingo.org/uuid`);
console.log(JSON.stringify(spec, null, 2));

Calling the API and displaying the results is trivial — just use the Fetch API and display the result as plain text:

// execCode sends an HTTP request according to the spec
// and returns the response as text with status, headers and body.
async function execCode(spec) {
    const resp = await sendRequest(spec);
    const text = await responseText(resp);
    return text;
}

// sendRequest sends an HTTP request according to the spec.
async function sendRequest(spec) {
    const options = {
        method: spec.method,
        headers: spec.headers,
        body: spec.body || undefined,
    };
    return await fetch(spec.url, options);
}

// responseText returns the response as text
// with status, headers and body.
async function responseText(resp) {
    const version = "HTTP/1.1";
    const text = await resp.text();
    const messages = [`${version} ${resp.status} ${resp.statusText}`];
    for (const hdr of resp.headers.entries()) {
        messages.push(`${hdr[0]}: ${hdr[1]}`);
    }
    if (text) {
        messages.push("", text);
    }
    return messages.join("\n");
}

const spec = {
    method: "GET",
    url: "https://httpbingo.org/uuid",
};

const text = await execCode(spec);
console.log(text);

Fetch API works in the browser, so there is no intermediate server involved. The only nuance is that the documentation must either be on the same domain as the API itself, or the API must allow cross-domain requests. But even if that's not the case, you can always proxy the requests — it's not too much work.

If you want an out-of-the-box solution, I've written a simple library that supports both JavaScript and Fetch API playgrounds (and some others):

codapi-js

Ideally, I'd like most documentation to be interactive. Not just API guides, but everything from algorithms (like hashing) to programming languages (like Go or Odin) to databases (like SQLite), frameworks and tools, and even individual packages.

And (shameless plug here!) I'm building a platform that allows just that — easily embeddable code playgrounds for documentation, online education, and fun. Check it out if you are interested:

codapi

And please try to write an interactive guide the next time you develop an API!

 Subscribe to keep up with new posts.