Share this letter & I’ll send you some rewards for the referrals.
No time for small talk today:
The following is the second of a premium 2-part newsletter... if you’re serious about API design in 2026, then this newsletter is for YOU.
On with part 2 of the newsletter:
===
Some of these API design concepts are foundational, and some are advanced. ALL of them are super useful to software engineers building scalable APIs.
Curious to know how many were new to you:
JSON Web Token,
Throttling,
Endpoint,
Request-Response,
Headers,
OpenAPI,
Timeouts,
Microservices,
WebSockets,
CORS,
HTTPS and TLS,
Filtering & Sorting,
Retries with Backoff,
Load Balancing,
Circuit Breaker,
Error Handling.
For each, I’ll share:
What it is & how it works (in plain English),
A real-world analogy (if I found one),
Tradeoffs,
Why it matters.
Let’s go!
§
[Webinar] How to stop babysitting your agents (Partner)
Agents can generate code. Getting it right for your system, team conventions, and past decisions is the hard part. You end up wasting time and tokens in correction loops.
More MCPs give agents access to information but not understanding. The teams pulling ahead use a context layer to give agents exactly what they need.
Join live on Oct 7 (FREE) to see:
Where teams get stuck on the AI maturity curve
How a context layer solves for quality, efficiency, and cost
Live demo: the same coding task with and without a context layer
(Thanks to Unblocked for partnering on this newsletter.)
§
18. JSON Web Token
JSON Web Token (JWT) is a signed token that contains claims about the caller.
Three parts, separated by dots.
Header names the algorithm.
Payload holds the claims: this is user 42, they expire at noon, and have the admin role.
Signature proves nobody altered the first two parts. The server verifies signature with a key it already holds, reads the claims, and answers the request without touching a session store.
The token carries its own proof, so verification costs no database call.
Analogy
A festival wristband.
It says which zones you may enter and which day it dies. Security reads it in a glance without radioing the ticket office. If you tamper with it, the seal breaks, so the guard knows.
Tradeoff
You cannot easily revoke a token.
Fire an employee at nine, and their token still opens doors until it expires. The payload is signed, not encrypted, so anyone holding the token can read every claim inside it.
So keep expiry short, keep secrets out of payload, and use a refresh token to renew.
Why it matters
Stateless authentication across microservices, and the access token most OAuth providers hand back.
Its structure is fixed by a public specification, which is why every language has a library for it. Pair it with Authentication, which is the job it exists to do.
19. Throttling
Throttling slows down a caller instead of turning them away.
Rate limiting & throttling get used interchangeably, and they do different things.
A rate limiter counts requests and rejects the ones over the cap with a 429. A throttle accepts the request and delays it. It queues the call, releases it when capacity frees up, or answers with a Retry-After that asks the client to come back shortly. The work still happens, just later and more slowly.
Rejection protects the server…Throttling protects the server & keeps the caller alive.
Analogy
A metered on-ramp to a motorway.
The light drips cars onto the carriageway one at a time. Nobody is sent home. The queue on the ramp is annoying, and it's why the motorway itself keeps moving at seventy.
Tradeoff
A queued request holds a connection, and enough of them exhaust the very resource you were protecting.
Plus, delay hides the problem, so a service that has been struggling for an hour looks merely sluggish. So cap the queue & alert on the delay rather than the errors.
Why it matters
Expensive operations, shared infrastructure, and any burst you would rather absorb than reject.
Envoy, Kong, and most message queues implement it. Pair it with rate limiting, and be explicit in your documentation about which one you are doing.
20. Endpoint
An endpoint is a uniform resource locator (URL) that points to one resource on your server.
It’s the address you aim a request at.
/usersis the collection./users/42is one user./users/42/ordersis that user’s orders.
The path names the resource & the method names the action; together, they define one operation. Get the path wrong, and you receive a 404 rather than the data you meant to ask for.
Read a good endpoint aloud, and it explains itself.
Analogy
A street address.
The number and the road identify one door out of a million. Post a letter to the wrong number, and it arrives somewhere real, which is worse than it not arriving at all.
Tradeoff
Endpoint names drift as the API grows.
One person writes /users, the next writes /getUser, and six months later your API reads like it was designed by a committee that never met.
Use plural nouns for collections and let the verb live in the method.
Why it matters
The first thing anybody learns about your API, and the thing they will curse if you got it wrong.
Every REST framework routes on them. Pair it with OpenAPI, which turns your endpoint list into documentation nobody has to maintain by hand.
21. Request-Response
The request-response cycle is the basic way clients & servers communicate. The client sends a request, and server returns a response.
Every call follows the same simple script.
A request names what you want and where: a method, a path, a few headers, a body when there is one to send. The server reads it, does the work, and answers with a status code and usually a body of its own. The exchange is synchronous, so the client holds the line open and waits for that single reply before it does anything else.
Send, wait, read the reply. That is the entire cycle.
Analogy
Posting a letter and waiting by the mailbox.
You write the address on the envelope, the details inside, and a return address on the back. Then you stand there. Nothing else happens until the reply lands.
Tradeoff
The client blocks and a slow server holds that connection, and work that takes two minutes leaves a user staring at a spinner. Nothing comes back until everything comes back.
Accept the request, return a 202, and finish the work in the background.
Why it matters
The foundation under REST, GraphQL, and almost every API you will build.
Its rules are set by the HTTP specification. Pair it with Webhooks and WebSockets, the two answers for when waiting is the wrong shape.
Want life to feel a little easier?
Ben Meer’s new book, How to Be Good at Life, is packed with practical systems for work, health, money, relationships, home, and more.
22. Headers
Headers are key-value pairs that carry information about a request/response, separate from the body.
They describe the message rather than contain it.
Content-Typesays what format the body is in.Authorizationcarries the credential.Cache-Controltells every cache along the path how long to keep a copy.Acceptstates what the client can understand.User-Agentnames the software that called.
Servers, proxies, and browsers all read them before touching the body.
Metadata belongs on the outside.
Analogy
The outside of an envelope.
Stamps, postcodes, fragile stickers, return address. Every courier along the route reads the outside and acts on it. None of them open the letter.
Tradeoff
Headers are easy to forget, which is why so many bugs end in a Content-Type nobody set.
Sizes are capped, casing rules surprise people, and a header stripped by a proxy fails silently rather than loudly. So log the headers you rely on, and set Content-Type explicitly every time.
Why it matters
Authentication, caching, versioning, compression, and cross-origin rules all ride on them.
Any browser’s network tab shows you every one. Pair it with Caching, which does nothing at all unless the headers say so.
23. OpenAPI
OpenAPI is a standard file that describes every endpoint, parameter, and response of your API in one place.
You write it in YAML/JSON, and machines read it.
From that one file, you generate interactive documentation, client libraries in eight languages, a mock server for the front end to build against, and a test suite that catches the day your response shape quietly changed. The specification becomes the contract, and the contract becomes the source of truth.
Write it once & stop maintaining documentation by hand.
Analogy
An architect’s blueprint.
The plumber, electrician, and joiner all work from the same drawing without ever meeting. Change the drawing, and everyone sees the change. Change the wall without changing the drawing, and somebody drills through a pipe.
Tradeoff
A hand-written spec drifts from the code within weeks, and a spec that lies is worse than no spec at all. The file grows long, and reviewing a two-thousand-line YAML diff is nobody’s favorite afternoon.
Generate the spec from your code, or test your code against the spec.
Why it matters
Public APIs, partner integrations, and any backend whose consumers sit in a different room.
Swagger UI, Redoc, and Stoplight render it. Pair it with API Versioning, since each version deserves its own contract.
24. Timeouts
A timeout is the longest a caller will wait for a response before giving up.
Without one, a caller waits forever.
Set a connect timeout for reaching the server and a read timeout for hearing back. When a downstream service hangs, your request fails in two seconds instead of hanging alongside it. This matters because every waiting request holds a thread, a connection, and a slot in your pool. Enough of them & your healthy service falls over next to the sick one.
A fast failure beats a slow hang.
Analogy
Hanging up after thirty seconds of ringing.
Nobody is home. Listening to the ringtone for an hour won't make them answer, and it stops you from calling anybody else.
Tradeoff
Set one too tight, and you kill healthy requests that were simply slow; then you retry them, and you have doubled the load on a service that was only having a bad minute.
Measure your ninety-ninth percentile, then set the timeout above it.
Why it matters
Every network call you make is the first line of defense against a cascading failure.
Every HTTP client has a default, and the default is usually wrong. Pair it with Circuit Breaker, which you reach for when timeouts stop being occasional.
25. Microservices
Microservices split one application into small, independent services that talk to each other over APIs.
Each service owns one job and its own data.
The orders service owns the orders table. The payments service owns payments. Neither reaches into the other’s database. They call each other’s APIs, deploy on their own schedule, and scale independently, so a search feature under load gets more servers without anybody touching checkout.
Independence is the whole point, and the network is the whole cost.
Analogy
A food court instead of one enormous kitchen.
Each unit runs its own stoves, staff, and menu. The pizza place closes for a refit, and everyone else keeps serving. Ordering a meal from three units means three queues.
Tradeoff
Every function call you used to make in memory is now a network call that can time out, arrive twice, or never arrive.
A bug spans four services and six log files. Keeping data consistent across services is a hard problem with no free answer. So start with a monolith, and split it when the pain is real, not theoretical.
Why it matters
Large systems, large organizations, and any codebase where two changes cannot ship without waiting for each other.
The patterns that make it survivable deserve an article of their own. Pair it with API Gateway and gRPC, which keep the pieces reachable and fast.
§
Reminder: this is a teaser of the subscriber-only newsletter, exclusive to my golden members.
When you upgrade, you’ll get:
High-level architecture of real-world systems.
Deep dive into how popular real-world systems work.
How real-world systems handle scale, reliability, and performance.
(If this newsletter has helped you become a better software engineer, consider subscribing to support my work.)