ADR-0026: LLM usage is metered in money, as credits pinned to a fixed reference price — never in raw tokens

  • Status: Accepted
  • Date: 2026-11-06
  • Scope: core (any product that caps or bills a user’s or a tenant’s language-model usage)

Context

A quota in tokens looks natural and is wrong as soon as tokens stop costing the same. Expenses
metered its assistant in “billable tokens” and saw the quota drift twice, with no number on screen
changing (expenses 0026-chat-quota-meters-cost-units):

  • once caching moved most of a turn into cache reads, metered tokens per turn fell about 3× while
    the cost of a turn stayed the same, so the monthly cap was worth several times the money it was
    sized for;
  • two models (a standard and a premium tier) fed one meter at different prices;
  • when the chat moved to a cheaper model, a credit tied to “the chat model’s input token” would
    have cut every plan’s money allowance by a third.

Decision

1. The unit is money

One credit is the price of one uncached input token of a fixed reference model, chosen once.
Each usage row is priced with its own model’s rates (uncached input, cache write by TTL, cache
read, output) and converted to credits at the reference price. A cheaper model simply costs fewer
credits for the same tokens.

2. Weights are derived from configured prices, and a missing price stops startup

No weight is typed in. A model the product can call without a price, or a missing reference
price, is a startup failure: a meter that weighs zero is no cap. Keep a retired model’s price
while its rows are inside a budget window.

3. The meter sums stored usage rows

Per-call usage (tokens per kind, model) is stored as it happens, and the meter sums it in the
database. Nothing derived is stored, so a re-pricing applies to the current window when it
deploys, and cost reports read the same rows.

4. Users see credits, not tokens

Clients name the unit as credits. An API that already exposes token-named fields keeps the names
and changes their meaning only when used and budget move together, so ratios stay correct;
renaming is a versioned contract change.

5. A turn is bounded by the loop, not cut off

The cap is checked before a turn starts. A running turn finishes; its size is bounded by the
maximum number of model calls per turn. A turn cut off mid-loop is a broken answer.

Consequences

  • The quota tracks spend whatever caching, model choice or pricing does.
  • Plan sizes are stated in money and can be compared with the plan’s price.
  • Accepted: the worst overshoot is one turn.
  • Any change to how usage is recorded (a new token kind, a new TTL) must reach the meter, or it
    under-counts.