AI search

Training data

pretraining corpus · model weights

Definition

Training data is the corpus a model learned from before anyone typed a prompt — books, web crawls, code, whatever the lab used. It is frozen at some cutoff. A rebrand, a crisis, or a new statute after that cutoff will not be “known” until retrieval or a later training run.

Earned media still matters here: journalism and Wikipedia are heavily represented in many crawls. That is not a reason to skip live RAG pages.

How it works

Assume the model’s biography of you is old. Publish a current entity page and earn fresh third-party URLs. When a chat product has browsing off, you are arguing with weights; when browsing is on, you are arguing with retrieval.

Worked example

The model still says you have 40 employees. You have 400. The about page and three 2026 trade profiles say 400. With retrieval on, answers update. With retrieval off, the 40 lingers.

How it differs

RAG fetches live pages. Training data is the frozen prior. Hallucinations can come from either, or from mixing them.

Common errors

Emailing a lab to “delete the training data.” Assuming one CNN hit in 2019 still defines the weights. Ignoring Wikipedia because “we do RAG now.”

Sources