"Walk me through how you would choose a model for a production AI feature. What factors would you consider, and when would fine-tuning be the right call?"

Model selection is the first architectural decision in any AI feature — and one of the most commonly botched. Deliver the full framework in 90 seconds.

Missing beat 1 signals shallow thinking; missing beat 5 signals over-reliance on public benchmarks.

1. Name the constraint first — latency, cost, or quality, before any model name
2. Match tier to constraint — frontier / mid-range / lightweight follows from the constraint, not brand preference
3. Exhaust prompting and RAG first — both faster to iterate, easier to reverse than fine-tuning
4. Fine-tune only with all 4 signals present — style drift, rare vocabulary, tight latency budget, very high volume
5. Validate with your own evals — leaderboard scores tell you where to start, not where to stop

"The first question I ask is: what's the binding constraint — latency, cost, or quality? That collapses the decision space immediately. For latency-sensitive tasks I start mid-range; for high-stakes reasoning with no latency pressure, frontier; for high-volume simple classification, lightweight — the tier follows from the constraint, not brand preference. Before any fine-tuning conversation, I invest seriously in prompt engineering — at least ten meaningfully different strategies — and RAG if the problem is factual accuracy, since both are faster to iterate and easier to reverse. I only evaluate fine-tuning if I have consistent style drift prompting can't hold, rare domain vocabulary, a latency budget a long system prompt can't meet, or very high volume on a narrow task — all four should apply. The final step is always running evals on my own data and my own task — a model that scores second on a general benchmark may outperform the leader on my specific domain."