AI search

Retrieval-augmented generation

RAG

Definition

Retrieval-augmented generation is the pattern most live answer engines use: fetch documents first, then write the answer with those documents in context. Aggarwal et al. described generative engines this way. The citation is supposed to be a breadcrumb back to a retrieved page. It is not always accurate.

Pretraining is the other half: facts the model already “knows” without fetching. Brand questions mix both. A rebrand last month will not be in training data; it has to be retrieved.

How it works

Make pages fetchable: indexable, clear, dated, with stable URLs. Put the number in text. Earn third-party pages that repeat it. Then audit citations by opening the URL, not by asking the model if it used you.

Worked example

A query about a new state privacy rule retrieves the legislature page, two law-firm memos, and a trade recap of your briefing. The answer cites two of four. Your owned explainer was blocked by noindex and never entered the pile.

How it differs

Grounding is binding a sentence to a retrieved span. Hallucination is a claim without support. Training data is the frozen corpus. RAG is the live fetch.

Common errors

Optimizing “for the model” with synonym stuffing. Assuming a cited URL was actually read. Noindexing the only page that has the table.

Sources