GraphQL and webhooks

A webhook is an HTTP request a service sends by itself to a set address when an event happens. GraphQL is an API with one address where the client picks fields.

When the server knocks on your door

Imagine a payment or an inquiry from a form is due to arrive. How do you find out? There are two ways. You can ask: every few minutes go to the service and say “well, has it come?”, and nine times in ten hear “no”. That is called polling, and it is wasteful and always late. Or you can ask the service to call you: “when it arrives, send me a message at this address”. Such a callback over HTTP is called a webhook.

It works like this. You tell the service the address of your handler (https://riverbend.example/hooks/request). When an event happens (a payment, an inquiry, a new commit), the service itself sends a POST request to that address, with a description of the event in JSON. Your handler reads it, does the job (for example, forwards the inquiry to the owner) and answers with 200. Payment notifications, chat messages and the rebuild of a site after an edit all work on the same principle.

A good handler follows three rules, and each of them was born of someone’s trouble.

Rule one: check that the request is genuine

The address of your handler is no secret: anyone can find it and send anything to it. If the handler believes the first request that comes along, an “inquiry” or a “payment” can be faked. So the service signs the request. The most common method is called HMAC: the service takes the body of the request and a secret shared with you, computes a hash (a checksum string) from them and sends it in a header. The handler computes the same itself and compares. They match: the request came from someone who knows the secret, and the body wasn’t changed on the way. They don’t: answer 401 and nothing more.

The signature is computed over the raw body, exactly as it arrived (if you first parse the JSON and write it out again, the spaces change and the signature won’t match). Compare signatures with a special function (timingSafeEqual) that always takes the same time: an ordinary string comparison tells an attacker, by how fast it answers, how many of the first characters he guessed. Some services (Telegram, for one) don’t sign at all: they send the secret itself, the one you chose when you set it up, in a header, and that is what you compare.

Try it by hand: change the body or the secret and see what happens to the signature. The browser computes it with real cryptography, but nothing is sent anywhere:

X-Signature: …

expects:  …
arrived:  …

And the secret lives outside the code. Whatever lands in a public repository is considered leaked for good, so the secret goes in the server’s environment variables (process.env.WEBHOOK_SECRET) or in a secrets store, and is changed if it may have leaked.

Rule two: answer quickly

A service waits for an answer only briefly, a few seconds. If you haven’t answered 2xx in that time, it decides the request didn’t arrive. So the handler first checks the signature, records the event and answers 200, and does the heavy work (sending a message, recalculating, calling another service) afterwards, in a queue for instance.

Rule three: a repeat must break nothing

If there is no answer, or it comes back as an error, the service repeats the request: after a minute, after five, after an hour, with growing pauses. So delivery works on an “at least once” principle, not “exactly once”. The same request can arrive twice: say you processed the event, but the answer got lost on the way.

So the processing must be idempotent: a repeated request must not repeat the effect. For that, every event has a number (id). The handler remembers which numbers it has seen, and simply acknowledges a repeated event (200) without doing anything. Without that, the client gets two inquiries and the shopper pays twice.

Where to send the inquiry

There are several ways to forward an inquiry to the owner: an email, a message in a chat app, a row in a spreadsheet. Slack has incoming webhooks: the handler sends a POST with the text to a secret address. That address is a secret just like the signature, so it doesn’t belong in the code. Telegram, popular in some countries, has a bot API where the handler calls sendMessage with the bot’s token, and that token is a secret too.

There is a legal side here as well. A visitor’s name and phone number are personal data, and a message sent to a chat app or an email service lands on servers outside your own hosting, and perhaps outside your country. Under the GDPR in the EU, and the UK GDPR in the United Kingdom, a service that handles the data for you is a processor: you need a contract with it, and if it sits in another country, the rules on international transfers apply. So every such channel belongs in your privacy notice, and is worth discussing with a lawyer. We return to the law in the part on quality and the law. For comparison, Russia goes further than the GDPR: its localization rule requires that the first recording of Russian citizens’ personal data be made on servers in Russia.

This is general information, not legal advice. For the UK, the Information Commissioner’s Office (ico.org.uk) explains the rules; elsewhere in Europe, ask your national data protection authority.

GraphQL: the client picks the fields

Alongside REST came another way to ask a server for data: GraphQL. It was invented at Facebook in 2012 (inside the company, for its mobile app) and opened up in 2015. The difference is that the API has one address (POST /graphql), and in the body of the request the client lists the fields it needs:

{
  items {
    title
    price
  }
}

and it gets exactly those, no more and no less. In REST the address /items returns every offering whole, even if the page needs only the title and price. If data from several related resources is needed, REST takes several requests and GraphQL takes one. The shape of the data is described by a schema with types, and documentation and checks are built from it.

Choose fields and compare the sizes of the response:

{
  items {
    title
    price
  }
}

{
 "data": {
  "items": [
   {
    "title": "First offering",
    "price": "£0.00"
   },
   {
    "title": "Second offering",
    "price": ""
   }
  ]
 }
}

GraphQL has a price too: it needs a separate layer on the server, it is harder to cache (every request goes to one address), and you must guard against queries that are too heavy (“give me everything about everyone”). So it is chosen when a project has several different clients (a site, a mobile app) that need different fields, and a lot of related data. For a list of a dozen services on a small business site, REST is simpler and enough.

Practice

You are on a site from the 2022 era, where sites talk to services. Set up the path of an inquiry from the form to the owner. A real receiver can’t run in the sandbox, so you write the handler’s code as text, as it would sit on a server, following the example in the lessons on the signature and repeats: in a pre block on the page. The checks look in it for the essentials: the secret from the environment, the HMAC signature, a safe comparison, the 401 answer, protection against repeats and a quick answer.

In two paragraphs, answer in words: what happens if the receiver is down, and when GraphQL is handier for your data and when REST. Name one of your items from the brief so that the reasoning is about your project.

What was wrong before

The main way to learn about an event was polling, and services exchanged data through SOAP and through files that somebody put on an FTP server on a schedule. When webhooks appeared, they were received with no signature (“nobody knows the address”), the secret was baked into the code and pushed to GitHub, and the “200” was sent only after heavy work, so services sent repeats and duplicate inquiries had to be sorted out by hand.

What is right has stayed: a callback instead of polling, a signature, a quick answer and idempotency. And GraphQL never pushed REST out: both live on, because they solve slightly different problems.