Ask five engineers which model to use for a new AI feature and you’ll get five model names, offered with roughly equal confidence, and roughly no shared reasoning behind any of them.
That’s the wrong first move. The right first move is naming a constraint, not a model — because the three things you actually care about, cost, latency, and quality, trade off against each other in a way that makes “just pick the best one” meaningless. You can’t maximize all three. You pick two to optimize and let the third flex.
What the triangle actually costs you
It’s easy to nod along with “there’s a trade-off” as an abstraction and much harder to feel it as a number. So here’s one: take a mid-sized AI feature — say, an internal research assistant fielding 2,000 queries a day, each one carrying roughly 600 tokens of prompt and context in, 250 tokens of answer out. Run that same workload through three tiers of model and price it out.
At the lightweight tier, that workload runs for a few dollars a day. At a mid-range tier, it’s roughly four times that. At a frontier tier, it’s roughly five times the mid-range number again — which means frontier ends up costing about twenty times what lightweight does, for the exact same requests. Nothing about the task changed between those three runs. Only the tier did.
That gap is the triangle made concrete. If quality is your binding constraint — a compliance answer that has real legal weight, say — that 20x multiplier might be trivially worth paying. If it isn’t, you’ve been quietly overpaying by a factor of twenty for months without a single stakeholder noticing, because nobody ran the arithmetic before the model got chosen.
Naming the constraint first
The question that collapses this decision isn’t “which model is smartest” — it’s: what’s the one dimension here that genuinely can’t flex?
If it’s latency, you’re looking at mid-range or lightweight, and the exact cutoff usually comes down to volume: high-volume, latency-critical traffic pushes you toward lightweight; moderate volume with a real latency floor still tolerates mid-range.
If it’s quality, and there’s no latency pressure forcing your hand, you start at frontier — a legal or medical use case is the textbook example, where the cost of a wrong answer dwarfs the cost of the API call that produced it.
If it’s cost, and the task is well-bounded — routing, classification, simple extraction — lightweight is very often not a compromise at all. It’s just the right tool, and frontier would have been the waste.
Notice what’s missing from all three of those: a specific model name. The tier follows from the constraint. The model within the tier is a much smaller, much later decision.
Where fine-tuning fits — and where it doesn’t
It’s tempting to treat “the model isn’t quite good enough” as a signal to fine-tune. It usually isn’t. Fine-tuning is the most expensive, least reversible lever available, and it only pays off when a fairly specific set of conditions line up together — not any single one of them in isolation. We’ve written the full decision framework for that question separately, including the three questions to ask before you commit and a real benchmark script. The short version: exhaust prompting, then RAG if the gap is factual rather than stylistic, and only consider fine-tuning once both of those are genuinely spent.
Tier selection and the fine-tuning decision are two different forks in the same tree. Confusing them is how teams end up fine-tuning a model when a better system prompt would have closed 90% of the gap in an afternoon — or, just as often, staying on a frontier model out of habit for a task a lightweight tier was already handling fine.
Treat it as arithmetic, not taste
The unglamorous truth is that model selection is one of the few AI engineering decisions that reduces almost entirely to a spreadsheet: your workload’s token shape, your provider’s per-tier pricing, and the one constraint you can’t compromise on. Run those three inputs through the framework above before you open a leaderboard, and the leaderboard becomes a tie-breaker instead of the whole decision.
Curious what the actual multipliers look like when you run them yourself, and how a benchmark project turns this into something concrete? The full chapter — including a hands-on tier-comparison exercise and the interview framing for this exact question — is part of the AI Transition path. If you want to see how this framework holds up under actual interview questions, we walk through the pattern here.