REST API design for business integrations: authentication, webhooks, retries and idempotency
Most integration failures are not about the happy path. This is a practical guide to designing APIs and webhooks that stay correct when requests fail, repeat or arrive late.
Connecting a store to a CRM, a payment provider, a courier or an email platform looks like plumbing, and on a good day it is. The difficulty is everything else: requests that time out, events delivered twice, systems that disagree about what happened. Designing for those cases is what separates an integration that works in a demo from one a business can rely on.
Start with the shape of the API
A REST API models things as resources with predictable addresses and uses HTTP methods to act on them. Keep it boring and consistent.
- Name resources as plural nouns, such as /orders and /customers, and use the HTTP method for the action: GET to read, POST to create, PUT or PATCH to update, DELETE to remove.
- Return the right status codes: 200 or 201 for success, 400 for bad input, 401 or 403 for authentication and permission problems, 404 for missing, 409 for conflicts, 429 for rate limits, 5xx for your own failures.
- Use one error format everywhere, with a stable machine-readable code and a human-readable message.
- Version the API from the start, for example /v1/, so you can change it without breaking existing integrations.
- Paginate list endpoints and let clients filter, so one request never returns an unbounded amount of data.
Authentication and access
Always use HTTPS. For simple server-to-server integrations, API keys are usually enough: a key identifies the caller and can be revoked on its own. When an application acts on behalf of users, use OAuth 2.0. WooCommerce's REST API, for example, uses consumer keys and secrets over HTTPS, and WordPress supports application passwords for the same purpose.
- Keep secrets in environment configuration, never in code, repositories or URLs.
- Give each key the narrowest permission that does the job, and one key per integration so a leak can be contained.
- Make rotation easy and log which key made which request.
Make writes safe to repeat with idempotency
Networks fail in the middle of requests. A client sends a payment or an order, loses the connection, and cannot tell whether it went through. If it retries, you must not charge or create twice. The standard solution is an idempotency key: the client sends a unique value with the request, and the server remembers the result for that key. A repeat with the same key returns the stored result instead of repeating the action. Payment APIs such as Stripe work this way, and it is worth building into any endpoint that creates something or moves money.
Webhooks: how to receive them well
Webhooks push an event to your system when something changes, such as an order being placed. They are efficient, but they come with guarantees that are weaker than people expect: they can be delayed, delivered more than once, and arrive out of order. A reliable receiver follows a few rules.
- Verify the signature. Most senders sign the payload with a shared secret, as WooCommerce webhooks do, so you can confirm the request is genuine before trusting it.
- Respond quickly with a success status, then do the real work in the background. Slow handlers cause timeouts and needless retries.
- Put the event on a queue and process it from there, so a spike or a failure downstream does not lose events.
- Store the event ID and ignore repeats, so processing twice has no effect.
- Do not assume order. Use timestamps or the current state of the resource, and be ready to fetch the latest version from the API.
Webhooks or polling?
Use webhooks where they exist, because you learn about changes immediately and avoid pointless requests. Keep a scheduled poll as a safety net: a periodic job that checks for anything that changed and reconciles it. Webhooks tell you quickly, polling guarantees you eventually catch what was missed.
Retries and rate limits
When you call someone else's API, expect failures and design the retry behaviour deliberately.
- Retry only safe or idempotent requests, and only on errors that are likely to be temporary: timeouts, 429 and 5xx responses.
- Back off with growing delays and some random jitter, so many clients do not retry in lockstep.
- Honour a Retry-After header when a 429 response includes one.
- Set a maximum number of attempts, then move the item to a failed list that a person can inspect and replay, instead of retrying forever.
Logging, monitoring and replay
An integration you cannot see into will fail silently. Log every request and event with an ID, the outcome and enough context to trace a single order across systems. Alert on failures and on queues that stop moving. Keep failed events so they can be replayed once the cause is fixed. This is the difference between hearing about a problem from your monitoring and hearing about it from a customer.
Document it and test the failure paths
Document the API for the next engineer: endpoints, authentication, error codes, webhook payloads and how retries behave. A machine-readable description such as OpenAPI makes this easier to keep accurate. Then test the unhappy paths on purpose: duplicate events, timeouts, a slow downstream system, an expired key. Those cases are where integrations break in production.
How I build integrations
I build integrations between WooCommerce, CRMs, email platforms, payment and courier services and custom applications. The common thread is the design above: signed webhooks, queued processing, idempotent writes, retries with limits, and logs that explain what happened. If you have systems that need to talk to each other reliably, describe them and I will propose how to connect them.
Frequently asked questions
What is an idempotency key?
An idempotency key is a unique value a client sends with a request so the server can recognise a repeat. If the same request arrives twice, for example after a timeout, the server returns the original result instead of performing the action again.
Should I use webhooks or polling to connect systems?
Use webhooks when the other system supports them, because you are told about changes as they happen. Keep polling as a fallback to catch missed events or reconcile data, since webhooks can be delayed, repeated or lost.
How should a webhook receiver be built?
Verify the signature, return a success response quickly, put the event on a queue and process it in the background. Store the event ID so repeat deliveries are ignored.
What authentication should a business API use?
Use HTTPS always. API keys suit simple server-to-server use. OAuth 2.0 suits access on behalf of users. Whatever you choose, keep secrets out of code and URLs, scope permissions narrowly and make keys easy to rotate.
How do I handle API rate limits?
Respect the limit and the Retry-After header when a 429 response arrives, back off with growing delays and jitter, and avoid retrying in a tight loop.