moflab notes

Backpressure in a hand-rolled queue

Oct 2026 · 4 min read

A small service consumed jobs from a table and pushed them to an HTTP endpoint. Latency grew, so we added workers. Throughput went down.

The reason was not obvious: with more workers, more requests were in flight, and the endpoint answered slower. The queue kept handing out jobs, memory grew, and the process spent its time in the garbage collector rather than in the requests.

The fix was a bounded buffer of 64 items in front of the workers. When it was full, the reader stopped reading. That single change took the p99 from 2.4 s to 220 ms and made the memory graph flat.

Lesson recorded for next time: a queue between two components is a promise that you can absorb an unbounded amount of work. It is not. Buffers exist to smooth bursts, and they need a limit and a way to say stop.