Glossary
The words, after the scene.
Each term is named in a lesson only once you have watched it happen. Here they all are, in plain words.
- Acknowledgementalso ack
- A small reply that only says “got it”. Without one, a sender can't know a message arrived.Shown in N2 Acknowledgements
- AIMDalso additive increase, multiplicative decrease, slow start
- Grow the sending window by one per round trip; halve it on a loss. It shares a link fairly.Shown in N9 Congestion control
- Anycastalso geo routing, latency routing
- One address announced from many regions: the network delivers each user to the nearest one that answers.Shown in N16 Nearest region and failover
- At-least-once delivery
- A message is handled until someone confirms it: nothing is lost, but some run twice.Shown in N14 At-most-once, at-least-once, exactly-once
- At-most-once delivery
- A message is handled once or not at all: nothing is repeated, but some can be lost.Shown in N14 At-most-once, at-least-once, exactly-once
- Availabilityalso uptime, SLI
- The share of requests answered well and on time, over a long window such as a month.Shown in F6 Nines, SLOs and error budgets
- Back-pressurealso backpressure, flow control
- A full queue pushing back on its producer, which waits instead of piling on more.Shown in N15 Backpressure
- Backlog
- Work waiting on a queue for a worker. It grows when jobs arrive faster than they are done.Shown in A1 Message queues
- Bandwidthalso throughput of a link, Mb/s
- How many bits a link sends each second. A big answer takes size ÷ bandwidth to send.Shown in N3 Latency vs bandwidth
- Bandwidth-delay productalso BDP
- Bandwidth times round trip: how much data is on the wire at once when it is kept full.Shown in N3 Latency vs bandwidth
- Batchingalso batch query, pipelining
- Sending many items in one request, so they share one round trip instead of paying one each.Shown in N13 Pipelining and batching
- Bulkheadalso pool isolation
- A separate pool of workers for one kind of work, so a slow dependency can't take them all.Shown in R5 Bulkheads
- Cache hit
- The cache had a copy, so the answer came back without asking the database.Shown in C1 Caching
- Cache miss
- The cache had no copy: the read goes on to the database, and the answer is kept for next time.Shown in C1 Caching
- Cache penetration
- Requests for things that don't exist: they miss every time and all reach the database.Shown in C5 Misses that never hit
- Cache stampedealso thundering herd
- Many requests missing on the same popular item at once and rushing the database together.Shown in C4 Cache stampede
- Cache warmingalso pre-warming, priming
- Loading a cache with what users will ask for before it takes their traffic.Shown in R7 Cold caches and thundering herds
- Canary releasealso canary deployment
- Sending a small share of traffic to a new version first, so a bad one hurts only a few users.Shown in O1 Canary rollouts
- Capacityalso throughput ceiling
- The most a part can finish each second: its workers divided by the time each request takes.Shown in F1 Latency & throughput
- Cascading failurealso domino failure
- One part fails, its load lands on the others, and they fail too, one after another.Shown in R6 Cascading failures
- Circuit breaker
- Stops calling a failing dependency for a while after repeated failures, then tests it with one call.Shown in R2 Circuit breakers
- Compare-and-setalso CAS, conditional write, optimistic locking
- A write that succeeds only if the value is still what it was when read; otherwise it is refused.Shown in D10 Locks and double booking
- Congestionalso network congestion
- More traffic than a shared link can carry: it queues in the link's buffer, then overflows and is lost.Shown in N9 Congestion control
- Congestion collapse
- A full network carrying mostly resent copies of late messages, so almost nothing useful gets through.Shown in N9 Congestion control
- Connection poolalso pool, PgBouncer
- A few connections kept open and lent out to requests, instead of opening one per request.Shown in D5 Connection pools
- Consensusalso Raft, Paxos, majority vote
- Copies agreeing by majority vote on one leader and one order of writes, so a cut can't make two.Shown in D9 Leader election (Raft)
- Consistent hashingalso hash ring
- Placing keys and nodes on a ring, so a node joining or leaving moves only its neighbours' keys.Shown in D4 Consistent hashing
- Consumer group
- Readers sharing a log's partitions, one reader per partition, rebalancing when one leaves.Shown in A7 Partitions, ordering and consumer groups
- Consumer lag
- How far a log's readers are behind its writers: events waiting to be read.Shown in A7 Partitions, ordering and consumer groups
- Dead-letter queuealso DLQ
- A side queue for messages that failed too many times, kept for a person to inspect.Shown in A5 Poison messages and dead-letter queues
- Deadlinealso deadline propagation, timeout budget
- The moment a request stops being worth answering. Passed along, every hop can drop work past it.Shown in N8 Timeouts and deadline propagation
- DNSalso Domain Name System, resolver
- The internet's phone book: it turns a name like example.com into the address of a server.Shown in N5 DNS: finding the server
- Double bookingalso race condition, lost update
- Two requests each checking that something is free, then both taking it, because neither saw the other.Shown in D10 Locks and double booking
- Drainingalso graceful shutdown, connection draining
- A server leaves the rotation, finishes the work it has, and only then stops.Shown in O2 Graceful shutdown and draining
- Dual write
- Writing the same change to two systems one after the other. If the second write fails, they disagree.Shown in A6 The outbox pattern
- Edgealso CDN edge, point of presence
- A cache close to users. Many of them together make a content delivery network (CDN).Shown in C2 Content delivery networks
- Error budgetalso SLO budget
- The failures a target still allows: 99.9% over a month leaves about 43 minutes.Shown in F6 Nines, SLOs and error budgets
- Evictionalso LRU
- A full cache drops the copy it used least recently, to make room for a new one.Shown in C1 Caching
- Exactly-once processing
- At-least-once delivery plus a receiver that ignores repeats, so each message has its effect once.Shown in N14 At-most-once, at-least-once, exactly-once
- Existence checkalso Bloom filter
- A compact list of every id that exists. It answers “no” at once, and “maybe” when unsure.Shown in C5 Misses that never hit
- Exponential backoffalso backoff
- Wait before retrying, and wait twice as long after each failure.Shown in A3 Retries & backoff
- Failoveralso promotion, switchover
- Making a replica the new primary when the primary dies, so writes can carry on.Shown in D6 Primary failover
- Flow controlalso receive window, rwnd
- The receiver telling the sender how much room it has, so it is never sent more than it can hold.Shown in N6 Windows and flow control
- Ghost request
- Work still waiting in a server's line for a user who already gave up. Serving it helps nobody.Shown in N8 Timeouts and deadline propagation
- Handshakealso TCP handshake, SYN/ACK
- The round trip that opens a connection before any request can travel on it.Shown in N4 Handshakes and keep-alive
- Head-of-line blockingalso HOL blocking
- Everything in line waiting behind the one at the front that is slow or lost.Shown in N10 Head-of-line blocking and multiplexing
- Headroomalso N+1 capacity
- Spare capacity, so losing one server leaves enough for everyone.Shown in R6 Cascading failures
- Health checkalso probe, ping
- A small request a balancer sends each server to see if it still answers.Shown in F4 Health checks & failover
- Heartbeatalso keepalive, ping/pong
- A small message sent on a timer so silence becomes a signal: no answer means something is wrong.Shown in N12 Heartbeats and keepalives
- Hedged requestalso speculative retry, backup request
- A second copy of a request, sent once the first is late. The first answer wins; the other is cancelled.Shown in R8 Hedged requests: beating tail latency
- Idempotency key
- A unique id a client sends with an operation and reuses on every retry, so the server does it once.Shown in A4 Idempotency keys
- Indexalso B-tree
- A sorted structure that points straight at rows by a column's value. Reads get fast; writes do a little more.Shown in D1 Indexes
- Invalidationalso purge
- Deleting a cached copy as soon as the source changes, so the next read fetches it fresh.Shown in C3 Expiry & stale data
- Jitter
- A random part of each wait, so clients that failed together don't all retry at the same instant.Shown in A3 Retries & backoff
- Keep-alivealso persistent connection, connection reuse
- Keeping a connection open after an answer, so the next request skips the handshake.Shown in N4 Handshakes and keep-alive
- Late eventalso out-of-order event, watermark
- An event that arrives after its window closed. Waiting longer counts it, but delays every total.Shown in A8 Windows and aggregation
- Latencyalso delay, ping
- How long the first bit takes to arrive: set by distance and the route, not by link speed.Shown in N3 Latency vs bandwidth
- Latency targetalso SLO
- The response time a system promises. A request still going after it runs late, drawn amber.Shown in F1 Latency & throughput
- Layer 4 balanceralso L4, network load balancer, TCP proxy
- A balancer that forwards whole connections without reading them: fast, but blind to what's inside.Shown in N11 L4 vs L7: balancers, proxies, gateways
- Layer 7 gatewayalso L7, API gateway, reverse proxy, application load balancer
- A front door that reads every request and routes it by path, header or user, at a small cost.Shown in N11 L4 vs L7: balancers, proxies, gateways
- Leaderalso primary
- The one copy that decides: every write goes through it, and the others follow.Shown in D7 Consistency vs availability
- Load multiplieralso retry amplification
- Attempts sent for every real request. Each retry adds one, so retries push it above ×1.Shown in A3 Retries & backoff
- Load sheddingalso admission control
- Turning some requests away at once when overloaded, so the ones you keep are still served on time.Shown in R4 Load shedding and priority
- Multiplexingalso HTTP/2 streams
- Many requests sharing one connection at once, their pieces interleaved.Shown in N10 Head-of-line blocking and multiplexing
- N+1 problemalso N+1 queries
- Fetching a list one item per call: each item costs a whole round trip, one after another.Shown in N13 Pipelining and batching
- Negative caching
- Keeping a “not found” answer for a short while, so the same missing id isn't looked up again.Shown in C5 Misses that never hit
- p95also 95th percentile
- The time 95 of every 100 requests beat. Said plainly: the slowest 1 in 20 take this long or longer.
- Partitionalso shard of a log
- One slice of a log. Events with the same key always land in the same partition, in order.Shown in A7 Partitions, ordering and consumer groups
- Poison messagealso poison pill
- A message that fails every time it's handled, so retrying it never helps.Shown in A5 Poison messages and dead-letter queues
- Pollingalso short polling
- Asking a server again and again whether anything new has happened. Most answers say no.Shown in N7 Long polling, SSE and WebSockets
- Publish/subscribealso pub/sub
- Announcing an event once; every service that listens gets its own copy.Shown in A2 Publish / subscribe
- Queuealso waiting line, backlog
- Requests waiting their turn because every worker is busy. Waiting is most of what “slow” means.Shown in F1 Latency & throughput
- QUICalso HTTP/3
- A transport over UDP where each stream is delivered on its own, so one loss stalls only its stream.Shown in N10 Head-of-line blocking and multiplexing
- Quorumalso W and R, majority
- How many copies must answer before a write or read counts. W + R > N means they always overlap.Shown in D8 Quorum reads and writes
- Reconnect stormalso thundering herd of reconnects
- Every client reconnecting in the same instant after a server drops them, swamping whatever is left.Shown in N7 Long polling, SSE and WebSockets
- Replicaalso read replica, secondary
- A copy of a database that takes the primary's writes a moment later and answers reads.
- Replication lag
- The moment between a write landing on the primary and its copy reaching a replica.Shown in D2 Read replicas
- Requestalso call, HTTP request
- A message asking a server for something. It has to travel there before any work starts.Shown in N1 A request and a response
- Request coalescingalso single-flight
- Letting one miss fetch an item while the others for the same item wait for its answer.Shown in C4 Cache stampede
- Responsealso reply, answer
- The server's message back with the answer. It travels the same distance home.Shown in N1 A request and a response
- Retransmissionalso resend
- Sending a message again because its ack didn't arrive in time.Shown in N2 Acknowledgements
- Retry budget
- A cap on retries, such as one for every ten requests, so retries can't swamp a struggling service.Shown in A3 Retries & backoff
- Rollback
- Going back to the previous version. Automatic when a canary's errors pass a limit.Shown in O1 Canary rollouts
- Round robin
- A balancer giving each server the next request in turn, whatever it is doing.Shown in F3 Load balancing
- Round tripalso RTT, round-trip time
- Out and back across a link: twice its one-way time, before any work is counted.Shown in N1 A request and a response
- Sagaalso compensating transaction
- A multi-step job where each step has an undo, run in reverse when a later step fails.Shown in A9 Sagas: multi-step work that fails halfway
- Sliding windowalso TCP window, in-flight limit
- How many messages a sender may have out unanswered at once. Each answer lets the next one go.Shown in N6 Windows and flow control
- Split brainalso two leaders
- A cut network leaving a leader on each side, both taking writes: two versions of the truth.Shown in D9 Leader election (Raft)
- Stale read
- An answer from a copy that is older than the latest write.Shown in C3 Expiry & stale data
- Stateless server
- A server that keeps nothing about users between requests, so any server can serve anyone.Shown in F5 Stateless servers and sessions
- Sticky sessionalso session affinity
- A balancer sending each user to the same server every time, usually because their session lives there.Shown in F5 Stateless servers and sessions
- Stop-and-waitalso one in flight
- Sending one message and waiting for its answer before the next: one message per round trip.Shown in N6 Windows and flow control
- Table scanalso full scan
- Reading every row of a table to find the ones you want. Its cost grows with the table.Shown in D1 Indexes
- Tail latencyalso p99, stragglers
- The slowest few percent of responses. When a page waits on many servers, the tail becomes the usual case.Shown in R8 Hedged requests: beating tail latency
- Thundering herdalso cold start, restart storm
- Everyone acting at the same instant, like every read missing an empty cache: a spike nothing was sized for.Shown in R7 Cold caches and thundering herds
- Time to livealso TTL, expiry
- How long a cached copy may be used before it must be fetched again.Shown in N5 DNS: finding the server
- TLSalso HTTPS, SSL, encryption
- Encryption agreed when a connection opens: one more round trip with TLS 1.3, two with TLS 1.2.Shown in N4 Handshakes and keep-alive
- Token bucketalso rate limit
- Each client earns tokens at a steady rate up to a small store; each request spends one.Shown in R1 Rate limiting
- Transactional outboxalso outbox pattern
- Saving an event in the same transaction as the data, then relaying it, so it can't be lost.Shown in A6 The outbox pattern
- Visibility timeout
- How long a queue hides a job it handed out. Unacked by then, the job is handed out again.Shown in N14 At-most-once, at-least-once, exactly-once
- WebSocketalso server push, server-sent events, long polling
- A connection kept open so the server can send news the moment it has some.Shown in N7 Long polling, SSE and WebSockets
- Windowalso tumbling window, time bucket
- A stretch of time, like each second, whose events are added up together and written once.Shown in A8 Windows and aggregation
- Workeralso thread, connection
- One request a server can work on at a time. A server has a fixed number, drawn as sockets.Shown in F1 Latency & throughput
- Write-behindalso write-back
- A cache that confirms a write at once and saves it to the database later, in batches.Shown in C6 Write-through, write-behind, cache-aside
- Write-through
- A cache that saves each write to the database before confirming it, and keeps the new value.Shown in C6 Write-through, write-behind, cache-aside