What is fine-tuning and when should I fine-tune a model instead of prompting?

Try prompting first, then retrieval, and only fine-tune when you've hit a specific limit that neither solves. Fine-tuning is the most expensive and least flexible option, and it's reached for far too early — usually to solve a problem it doesn't actually address. **What fine-tuning does**: takes a pre-trained model and continues training it on your examples, adjusting the weights so it internalises a behaviour, style or format. What it's genuinely good at: - Teaching a consistent output *format* or *style* that's hard to specify in words - Domain-specific tone and terminology - Improving reliability on a narrow, repetitive task - Reducing prompt length (and therefore cost and latency) by baking instructions into the weights **What fine-tuning is bad at — and this is the crucial misconception**: adding knowledge. If you fine-tune on your company's documents hoping the model will answer questions about them, you'll get a model that adopts the *style* of your documents while still hallucinating the *contents*. Facts don't reliably stick this way. For knowledge, you want retrieval. The escalation ladder, in order: 1. **Better prompting.** Clearer instructions, explicit format specification, and especially few-shot examples in the prompt. A large share of 'we need fine-tuning' turns out to be 'we needed three examples in the prompt'. Cheapest, instant to iterate. 2. **Retrieval (RAG).** If the problem is 'the model doesn't know our specific information', fetch the relevant documents and put them in the context. This is the correct solution for knowledge, it's updatable without retraining, and it lets you cite sources — which also reduces hallucination. 3. **Better decomposition.** Break one hard task into several reliable steps with checks between them. Often fixes what looks like a capability problem. 4. **Fine-tuning.** When you've done the above and still need consistent behaviour at scale, or you need to cut cost and latency on a high-volume narrow task. The costs to weigh honestly: you need a genuinely good dataset (hundreds to thousands of quality examples — and creating them is the real work), plus training cost, plus a hosting story, plus the fact that you're now pinned to a base model while better ones keep being released. That last point is underrated: teams that fine-tuned heavily have repeatedly found that the next general model beat their specialised one with a plain prompt. The rule of thumb: prompt for behaviour, retrieve for knowledge, fine-tune for consistency at scale — and always build an evaluation set first, so you can tell whether any of it actually helped.