RAG versus fine-tuning is the architectural question that shows up in every AI project conversation right now. The framing is usually wrong from the start — they get presented as alternatives when they are actually different tools solving different problems. Product teams reach for fine-tuning because it sounds more sophisticated, then spend six months building something that RAG would have solved in three weeks.

This piece is a decision framework. Clean definitions of what each approach actually does, the cost and engineering trade-offs, the signals that tell you which one fits your problem, and the cases where you should use both. No PhD in machine learning required — but the distinctions do matter.

What They Actually Do

A language model is trained on a large corpus of text. After training, the model’s weights contain a compressed, lossy representation of everything it was trained on. When you send the model a prompt, it generates a response by applying those weights to the input.

That framing matters because it shows where RAG and fine-tuning operate.

RAG adds a lookup step before the model runs. Your system retrieves relevant information from a store of your data — documents, database rows, API responses — and includes that information in the prompt sent to the model. The model is unchanged; the information is injected at query time. The system of record for your data stays in your retrieval layer. Our RAG pipelines guide covers the engineering of this in depth.

Fine-tuning modifies the model’s weights by continuing training on your specific data. The result is a new version of the model that has absorbed the behaviour, patterns, or knowledge from your training examples into its weights directly. Once the model is fine-tuned, you run inference against your custom model; no retrieval layer sits between the prompt and the response.

RAG gives a model access to information. Fine-tuning changes what the model is.

The Four Dimensions That Matter

When choosing between the two, four dimensions separate them clearly.

Freshness of information

RAG retrieves at query time, so information is as fresh as the retrieval store. Update a document in your CMS and the next query sees the new content. No retraining, no redeployment. For use cases where information changes frequently — product catalogues, support documentation, pricing, inventory — this is decisive.

Fine-tuning bakes knowledge into weights at training time. Updating fine-tuned knowledge means retraining, which is slower and more expensive. For stable, well-defined knowledge that changes rarely (brand tone, domain vocabulary, compliance language), this is fine. For anything that changes faster than quarterly, it is a maintenance burden.

Traceability and citations

RAG makes it easy to cite sources. The retrieval layer returns specific documents; the generation layer can quote or reference them. Auditing why the model said something is straightforward — you look at what was retrieved. For use cases where traceability matters (regulated industries, customer-facing support, enterprise search), this is a significant advantage.

Fine-tuning absorbs training data into the model’s weights in a way that cannot be traced cleanly. Asking “why did the model say that” becomes an interpretability problem. The answer is woven into the weights, not attached to a source. For many business contexts this is unacceptable.

Cost profile

RAG costs split into build (£5,000 to £30,000 for a focused pipeline) and run (embedding costs, vector store hosting, LLM inference — usually a few hundred to a few thousand pounds per month for mid-market volume). The LLM is rented from a provider at per-token pricing; you pay for what you use.

Fine-tuning costs split into training (£20,000 to £100,000 for a focused project, depending on data curation and iteration), hosting (the tuned model sits on your chosen provider, with associated hosting fees), and inference (per-token pricing, often similar to the base model). The training cost is upfront; it pays off if the model will be used heavily enough to amortise the build.

For most business use cases, RAG wins on total cost. Fine-tuning earns its investment in high-volume, narrow-task scenarios — think a tuned classifier running at millions of queries per month — where inference cost dominates and the tuned model is smaller or cheaper per query than the general-purpose model.

Behavioural consistency

Fine-tuning shines when behaviour needs to be consistent across every response. Structured output formats. Specific tone and voice. Domain-specific vocabulary. Classification labels that need to match an internal taxonomy. In these cases, a fine-tuned model applies the trained behaviour reliably; RAG-plus-prompting leans on the model’s general capability, which drifts.

RAG shines when content needs to vary correctly based on what was retrieved. Answering a customer’s specific question with their specific order history. Summarising a document that is different every time. Generating recommendations based on fresh inventory. In these cases, the work is in the retrieval, not in shaping the model’s behaviour.

The Decision Framework

Four questions to work through in order.

Question 1: Is the gap knowledge or behaviour?

If the model’s problem is “it does not know our stuff” — product details, internal policies, recent events, specific data — the gap is knowledge. RAG solves knowledge gaps.

If the model’s problem is “it does not speak the way we need it to” — wrong format, wrong tone, wrong terminology, inconsistent classification — the gap is behaviour. Fine-tuning solves behaviour gaps, but so does careful prompt engineering much of the time.

Most teams misidentify their gap. The symptom is usually “the model is not good enough for our task.” The question is whether the missing piece is information or disposition.

Question 2: Have you exhausted prompt engineering?

Before reaching for fine-tuning, work the prompt. Language models are surprisingly malleable through prompting alone. Few-shot examples, structured output formats, clear role specification, and chain-of-thought prompting can move model behaviour significantly without any training.

A reasonable rule: if you have not spent at least two weeks iterating on prompts and evaluating systematically, fine-tuning is premature. Prompt work is cheap, fast, and often sufficient. Our AI strategy for mid-market businesses guide covers how to build evaluation into AI projects from the start.

Question 3: How often does the relevant information change?

Information that changes weekly or faster is a RAG problem. Product catalogues, inventory, pricing, support docs, news, customer data — all change continuously. Fine-tuning information that changes frequently is an operational tar pit.

Information that changes rarely — brand voice, regulatory language, domain vocabulary, stable taxonomies — is potentially a fine-tuning candidate if the behavioural consistency argument also applies.

Question 4: Does inference cost dominate the budget?

For high-volume narrow tasks, inference cost can become the primary expense. A classification task running at 10 million queries per month, a product tagging pipeline, a content moderation system — these scale to the point where a smaller fine-tuned model can be cheaper than repeatedly calling a frontier model. If you are in this range and the task is narrow, fine-tuning a small specialised model pays off.

For lower-volume tasks (under a million queries per month) or broader tasks (where the range of responses is wide), inference cost is rarely the bottleneck, and the flexibility of RAG is worth more than the inference savings of fine-tuning.

Common Scenarios Walked Through

Five realistic project types, each with the right answer.

Customer support assistant with access to product docs

Answer: RAG.

The knowledge (product documentation, support articles, order history) changes constantly. Traceability matters — customers and auditors need to see sources. Volume is typically moderate. RAG solves this cleanly; fine-tuning would be expensive, opaque, and outdated within weeks.

Internal search across years of company documents

Answer: RAG.

Vast corpus of documents, constantly evolving, specific answers depend on specific documents. This is the canonical RAG use case. Fine-tuning on internal documents would absorb a tiny fraction of the corpus into weights, produce unverifiable responses, and miss new documents entirely.

Consistent brand voice for marketing copy generation

Answer: Fine-tuning (plus RAG if needed).

Behaviour consistency is the primary goal: every response needs to sound like the brand. Fine-tuning on a curated corpus of approved brand copy bakes tone into the model. If the generated copy needs factual grounding (specific product claims, recent campaigns), add RAG on top.

Answer: Fine-tuning.

Narrow task, well-defined taxonomy, high volume, behaviour consistency is critical, and accuracy matters more than flexibility. Fine-tune a small specialised model on a curated training set. Validate rigorously against a held-out test set before production.

Product recommendation engine powered by an LLM

Answer: RAG with prompt engineering.

Inventory, prices, and customer preferences are fresh data. The model needs access to the current state of the world. Fine-tuning is wrong — the information is precisely what changes. Retrieve the relevant products and context, then prompt the model to generate the recommendation.

The Combined Architecture

For many production AI systems, the best answer is both. Fine-tune the model to adopt consistent tone, format, and domain behaviour. Use RAG to inject the specific facts each response needs.

This combination underlies many serious production AI systems. The fine-tuned model becomes a reliable instrument that speaks the right way; retrieval supplies the content to speak about. Neither approach alone is enough; the combination produces systems that are reliable in tone and grounded in data.

The engineering cost is higher — you are running both a fine-tuning project and a RAG pipeline — but for high-stakes production use cases, it is often justified. Typically these are systems operating at scale with tens of thousands of customer interactions per month and specific brand or regulatory requirements that prompting cannot satisfy.

What to Avoid

Three patterns show up repeatedly in AI projects that go badly.

Fine-tuning to teach facts. Teams regularly fine-tune models on internal documents, expecting the model to “know” that content. The result is a model that has absorbed a lossy, distorted impression of the data and cannot cite or verify any of it. Facts belong in retrieval, not in weights.

Fine-tuning too early. Reaching for fine-tuning before serious prompt engineering has been tried. Most teams that fine-tune could have achieved the same result with better prompts and fewer resources. Start with prompts, escalate to RAG, consider fine-tuning only when both have been exhausted.

RAG with no evaluation. Building a retrieval pipeline and shipping it without a systematic evaluation framework. The result is a system nobody trusts because nobody knows when it is wrong. Evaluation is the discipline that separates RAG projects that deliver from ones that quietly rot. Our piece on what an AI agency actually is covers evaluation as a core capability in more detail.

Pragmatic Starting Points

If you are scoping an AI project in 2026, three pragmatic recommendations.

Start with RAG. It is cheaper, faster to iterate, more traceable, and easier to maintain. Unless you have a specific reason to fine-tune, RAG is the default.

Build evaluation early. Before shipping anything, define what good looks like and create a test set that measures it. Eval-first AI projects ship useful systems; eval-later AI projects ship demos.

Be honest about engineering capacity. Both RAG and fine-tuning are engineering-heavy in production. Teams without dedicated AI engineers usually underestimate the operational surface. A smaller, well-maintained system outperforms an ambitious one that rots.

We build production RAG pipelines, fine-tuned models, and hybrid systems for mid-market businesses. If you want to talk through a specific project — pragmatic scoping, sensible architecture, proper evaluation — get in touch. Or work through the AI readiness audit first to identify the highest-value AI investment for your business before committing to an architecture.