Measuring the Real Cost of an LLM Call: Tokens, Precision, and Honest Numbers

2026-10-12 · QuantID

aimachinelearningllmdatascience

TL;DR

The price per million tokens is the sticker, not the cost. The real cost of an LLM call depends on how many tokens you actually spend, the precision the provider serves at, how much the request waits, and whether the endpoint is even up. This post is a measurement checklist for putting a defensible number on an LLM call, written from the view of a team whose product is measurement.

The sticker is not the cost

Two providers can quote the same price per million tokens and cost you very different amounts. The gap hides in four places.

• Token spend. Reasoning models and verbose system prompts change how many tokens a task actually burns. The unit price is fixed, the unit count is not.
• Precision. A model served at a lower numeric precision is cheaper to run, and the provider may or may not tell you which precision you are getting. Precision is a cost and quality lever at the same time.
• Latency. First-token latency and tokens per second decide whether a call fits a user-facing budget. Public price pages rarely publish these.
• Availability. A cheap endpoint that returns a 503 under load has an effective cost of infinity for that request.

Precision deserves its own line

When you compare two deployments of the same model, the flags on the command line are not the whole story. The served weights can differ in precision even when the launch looks identical, and that changes both speed and output. If you are benchmarking, pin the precision you are actually measuring and state it. A fair comparison names the precision; an unfair one hides it.

A simple way to keep this honest is to treat precision as an explicit weight in your cost model rather than folding it silently into a single price number. State the assumption, then let readers adjust it.

Latency is a cost, not a vibe

First-token latency and tokens per second are measurable, and they belong in the cost story. The catch is that public APIs often do not expose them, so you have to call the endpoint yourself with the same prompt and record the numbers. One call is a sample, not a law. Report how many calls, and from where, because the answer moves with both.

A measurement checklist

Before you publish a cost number for an LLM call, confirm:

1. You counted actual tokens for a representative task, not just the unit price.
2. You named the serving precision you measured.
3. You measured first-token latency and throughput yourself, with the call count stated.
4. You checked availability under the load you care about, not at idle.
5. You separated numbers you measured from numbers you assumed, and labeled the assumptions.

If a comparison is missing any of these, it is a quote, not a measurement.

Why this matters now

As more products route across many models, the question is no longer which model is best in the abstract. It is which deployment gives the quality you need at a cost you can defend. That is a measurement problem, and measurement problems reward discipline over marketing.

Note on relationships: QuantID works with technology partners on model and serving research; where a partner is relevant we disclose it as a partner rather than an endorsement.

FAQ

Is price per million tokens useless? No. It is one input. The cost is price times actual token count, adjusted for precision, latency, and availability.

How do I measure precision if the provider does not say? Measure behavior and speed on fixed prompts, and state that the served precision is unverified when it is.

Why measure latency myself? Because most public APIs do not expose first-token latency, and it is decisive for user-facing budgets.

One call or many? Many. One call is a sample. State the count and the origin.

Further reading

The theme that generalizes: a cost number is only as good as the conditions attached to it. Name the precision, state the call count, separate measured from assumed, and the number becomes something you can defend.