AI Hallucination

An AI hallucination is content an AI model generates and presents as fact that is false or fabricated — a price, statute, statistic, citation, or event that does not exist in any source. The model is not lying; it is completing a plausible pattern with nothing behind it.

Most writing about hallucinations uses hypothetical examples. We can do better than hypothetical: our own pipeline produced one, we caught it, and the incident is why one of our build gates exists. This page defines the term through that documented record.

What does an AI hallucination look like in practice?

Like numbers that are more believable than the truth. On 2026-07-28, the first day of OEM-page scraping on our car-leasing build, an extraction model read a manufacturer page that carried exactly 1 national lease offer — $479 per month — and returned a ladder of offers spanning $249 to $399 per month. None of those figures appeared anywhere in the page text [our data].

The danger is the plausibility. A $249–$399 ladder looks more like a normal lease page than a single $479 offer does. A reviewer skimming for weirdness would have passed it; only checking each figure against the source caught it.

Why do AI models hallucinate?

Because generation optimizes for plausibility, not truth. A language model produces the most likely continuation of the text in front of it, and when the learned pattern is stronger than the evidence — lease pages usually list several offers at varied price points — the model completes the pattern instead of reporting the page.

The grounded-versus-parametric distinction explains part of the risk: answers built from training data alone, with no source retrieved, have nothing external to be faithful to — and Kevin Indig's 2026 analysis found 24% of ChatGPT answers are generated without fetching any live page (Growth Memo, 2026). The full distinction is defined under grounding.

But our incident adds the uncomfortable half of the lesson: retrieval does not guarantee fidelity. The correct page was directly in the model's context, and it fabricated anyway [our data]. Grounding shrinks the fabrication surface; only verification closes it.

The term is also worth keeping narrow. A real figure misstated from a real source is an error; a hallucination is generated content with no source behind it at all. The two are caught by different checks — errors fall to source-tracing and recomputation, hallucinations to verifying that the claimed source exists and says what the page says it does.

How do you catch hallucinations before they publish?

Mechanically, with gates that check output against sources — not editorially, by reading for tone. Three mechanisms from our production record [our data]:

Verbatim verification for extraction pipelines. Every figure headed for publication — payment, price, term, even image URLs — must appear word-for-word in the source text, or the item is dropped. The gate landed the same day as the price-ladder incident and has governed every scrape since.

Refute-first audits against a page's own cited sources. Instead of asking "does this read right?", the auditor tries to disprove each claim from the exact source it cites. On the same leasing build, this caught 1 fully hallucinated legal claim on a published-candidate page — a fabricated state tax regulation, cited to a source that says the opposite — before it went live [our data].

Site-wide claim audits with a committed ledger. Our insurance build's audit (2026-07-30) checked 445 claims and corrected 40, with 0 fabricated [our data]. That split is the honest base rate in our records: most AI-content errors are wrong — a real figure misstated — not invented. The full accounting across all 3 builds is in what fact-auditing 400+ pages actually catches.

Can you fix what an AI hallucinates about your brand?

Sometimes, and the fix depends on where the false answer comes from. When the answer is grounded — the engine retrieved and misused live pages — publishing a clear, extractable correction gives future retrievals something better to quote; the workflow is in how to fix what AI says about your brand.

When the answer is parametric — generated from training data with nothing fetched — no page you publish reaches it quickly. Corrections arrive with retraining, on a timeline nobody outside the model provider controls, and nobody can honestly promise otherwise. The practical move in the meantime is documentation: dated screenshots of the false answer, so you can tell drift from fix when the model eventually updates.

What is the rule for publishers using AI?

Verification before automation, installed before the first run — not after the first incident. Our gate exists because we ordered those steps wrong by one day [our data]; the fabrication arrived with the very first scraping session.

Said against our own interest as a team that builds AI-assisted publishing systems: if you cannot fund the verification layer, do not automate the publishing. An unverified pipeline manufactures liabilities at machine speed, and the plausible fabrications are precisely the ones nobody catches by skimming. Verified pipelines are the only kind we ship, and the only kind the operator's guide to generative engine optimization describes.