Inference cost

What is inference cost?

Inference cost is the compute cost of running an AI model each time it produces an answer, usually priced by the number of tokens it reads and writes. Training a model happens once. Inference happens every time someone uses it. Model providers publish separate prices per million input tokens and per million output tokens, and output tokens usually cost more.

Why it matters to a revenue leader

Inference cost behaves like the cost to deliver a service visit: it grows with every use. A tool that summarizes each account once a week costs little, while an agent that rereads full account histories on every request can cost far more than its license suggests. Prices per unit of capability have fallen fast: Stanford's 2025 AI Index reports that the price of GPT-3.5-level performance fell from $20 to $0.07 per million tokens between November 2022 and October 2024. For a PE-backed company watching margin, the open question is who pays as usage grows.

How to calculate it

A facility services company with 30 sellers runs an agent that prepares a renewal brief for 400 agreements a month. Each brief reads 50,000 tokens and writes 2,000. At $3 per million input tokens and $15 per million output tokens, each brief costs $0.18 and the month costs $72. If the agent reruns each brief five times as data changes, the month costs $360.