Most “the AI gave a weird answer” bugs aren’t model bugs. They’re missing-field bugs.
A prompt that works in a quick test and then misbehaves in production almost always has the same root cause: it was never structured to survive contact with real, varied input. The fix isn’t a smarter model — it’s a framework, and it’s smaller than most people expect.
Two layers, not one prompt
Every request to an LLM actually carries two distinct pieces of instruction, even when they arrive as one string. The system prompt is developer-controlled: it defines who the model is, what rules it follows, and what shape its output takes. It’s set once and applies to every call. The user message is the runtime payload — the specific task or question for this particular request.
Conflating the two is where a lot of prompt design goes wrong. Put per-request data in the system prompt and you’re rewriting static instructions on every call for no reason. Put the model’s rules in the user message and they compete for attention with whatever the user actually typed.
RCTF: the four fields that belong in a system prompt
Role — who the model is and how it should behave. Not flavor text; it sets the register and boundaries of every response that follows.
Context — the background information the model doesn’t have and can’t guess. Product names, plan tiers, internal terminology. Skip this and the model doesn’t fail loudly — it fabricates something plausible-sounding instead, which is worse.
Task — the precise operation to perform on the user’s input. “Classify this” and “summarize this” produce structurally different outputs; an ambiguous Task field leaves that choice to the model.
Format — the exact output structure. For anything feeding a parser, this is non-negotiable: field names, allowed values, structure.
Each field addresses a distinct failure mode. Drop Role and tone drifts unpredictably across calls. Drop Context and the model invents facts. Drop Task and it does something adjacent to what you wanted rather than what you asked for. Drop Format and — this is the one worth dwelling on — the output shape becomes unreliable in a way that’s easy to miss until it’s already broken something downstream.
The Format field, measured
Here’s what “unreliable” costs, concretely. Take a support-ticket classifier meant to return exactly three fields: category, urgency, summary. Ask it for that shape without ever demonstrating it, and run the same request repeatedly at a moderate sampling temperature. In a controlled simulation of this exact scenario, the output drifted to a different field-naming scheme — type/priority instead of category/urgency — in roughly 8 out of every 10 runs.
Anchor the identical request with two or three worked examples showing the exact shape you want, and that drift rate drops to zero across the same number of trials, same temperature, same conditions. Nothing about the model’s knowledge changed. What changed is that the shape was demonstrated instead of merely described.
That’s the entire case for few-shot prompting in one data point: it doesn’t teach new facts, it teaches shape, and shape is exactly what a downstream parser is betting on.
Zero-shot first, always
The instinct to reach for few-shot examples immediately is understandable but usually premature. Start zero-shot — instructions only, no examples. It’s cheaper to run and easier to maintain, since there’s nothing to keep in sync when the underlying task changes slightly. Move to few-shot only when zero-shot is getting the content right but the format wrong or inconsistent. If the content itself is off, more examples in the prompt won’t fix a Context gap.
Where fine-tuning actually fits
It’s worth being explicit about where this framework stops being the answer. Few-shot examples live in the prompt and cost nothing to change — swap the examples, redeploy instantly. Fine-tuning retrains the model’s weights on your dataset, which is slower, more expensive, harder to reverse, and simply unnecessary for the format-consistency problem above. RCTF plus a couple of good examples solves that problem completely. Fine-tuning earns its cost only for stable, high-volume, narrowly-scoped tasks where prompting has been genuinely exhausted first — not as a shortcut around writing a better prompt.
The parameters sit next to the prompt, not inside it
Two API parameters do real work alongside RCTF. Temperature at 0 makes output deterministic — the same input produces the same output every time, which is what any classifier or structured-extraction task needs. Higher temperature is for brainstorming, not parsing. max_tokens caps response length; set it too low and structured output truncates mid-field, too high and you’re paying for unused headroom. Neither parameter fixes a badly structured prompt — they tune behavior around a prompt that’s already correct.
The prompt gets you the right shape. The parameters get you the same shape, reliably, every time. Both matter, and confusing which does which job is its own quiet source of production bugs.