← Blog

engineering

A spend cap that only checks at the start is not a cap

Most metering stops at the request boundary. The expensive part of an LLM call happens after it.

18 August 2026 · By Ada Okafor, Engineering · 7 mins

A request arrives, you check the balance, you let it through. That is where most metering ends, and it is where the money starts.

A streaming completion can run for minutes. A caller with $0.40 left can start a request that costs $6.00, and every check you did happened before the first token came back. The balance was fine when you looked.

  check-at-the-start                 hold-and-accrue

  balance $0.40  ✓                   balance $0.40
        │                            hold    $6.00  ← worst case, taken first
        ▼                                  │
  dispatch ─────────────►              available = 0.40 − 6.00 < 0
        │                                  │
        │  tokens ──────────────►           └──► 402 before dispatch
        │  tokens
        │  tokens          $6.00 spent
        ▼
  settle: balance −$5.60   ✗ overrun
A check at the boundary sees a balance that is already stale.

The fix is not a bigger check at the start. It is a hold taken before dispatch, an accrual loop that watches the response as it arrives, and a socket that gets cut when the hold is exhausted, with a terminal frame that tells the client why, in the shape its protocol expects.

ts
// the loop that makes the cap real: accrue as chunks arrive, abort when the hold is gone
for await (const chunk of upstream) {
  accrued += costOf(chunk)

  if (accrued >= hold.amountMicros) {
    // a stream that just stops looks like a network failure, and a client that
    // cannot tell the difference will retry, which is the last thing we want here
    yield terminalFrame({ code: "spend_cap_exceeded", spentMicros: accrued })
    await upstream.cancel()
    break
  }

  yield chunk
}

await settle(hold, accrued) // hold → charge, atomically

That last part matters more than it sounds. A stream that simply stops is indistinguishable from a network failure, and a client that cannot tell the difference will retry, which is the one thing you do not want a caller at their limit to do.

Ready? Let's go.

Whether you want to examine the specific needs of your product, or go over the benefits of seams, we are here for you.