Fact-Auditing 400+ Pages: What Three Audits Caught

Everyone publishing with AI assistance claims their content is fact-checked; almost nobody publishes what the checking found. This page is the record for our 3 production builds — 400+ published, fact-audited pages — with the counts, the dates, and the specific errors, because the errors are the interesting part [our data].

The summary conclusion up front: every build needed an adversarial audit, and every audit found real defects our automated gates had missed.

Why fact-audit AI-assisted content at all?

Because an answer engine that quotes your page republishes your claims under your name, and a skeptical reader who catches one wrong figure discounts every other figure on the site. Google's people-first guidance frames the quality bar in the same terms — content that demonstrates reliability and first-hand depth (Google, helpful content guidance) — but the operational reason is simpler: we knew our pipeline was AI-assisted, so we assumed errors existed and went looking.

The audits are not spot checks. Each one works claim by claim: every statistic traced to an approved source or a first-party record, every derived number recomputed from scratch, every hedge ("roughly," "commonly") treated as an unverified claim to verify or cut. Each audit's ledger is committed to the build's repo [our data].

What did the three audits actually catch?

Three builds, three audits, three different worst finds [our data]:

The second audit also caught defects that were not factual at all: ordered lists rendering as run-on paragraphs, literal asterisks visible on 35 pages, and 58 meta descriptions over length — all invisible to gates that only read the source files [our data].

Which failure classes matter most?

Five distinct classes came out of the ledgers, and the cheapest-looking one is the most dangerous [our data].

Wrong-but-sourced figures. The real source exists; the number on the page misstates it. The "roughly $10,000" exemption whose regulation says $12,997 is the type specimen — plausible, hedged, and wrong by 30%.

Propagated math. One computed figure goes wrong once and gets reused. Of 461 recomputed payment figures, exactly 1 was wrong — but a wrong figure that propagates is wrong everywhere it lands.

Zombie claims. A claim gets retired, then returns: writing agents matching older exemplar pages faithfully reproduced a retired convention onto 6 new pages. The claim was dead in the style guide and alive in the corpus.

Render-layer defects. The source is correct and the built page is broken — run-on lists, stray asterisks, over-length metas. Source-level review can never see these.

Outright fabrication. The rarest and worst: a state tax regulation that does not exist, cited to a real source that says the opposite. It read as the best-sourced sentence on the page. The mechanics of this class get their own definition at AI hallucination.

What did each catch change in the system?

Every error became a rule, which is the actual return on an audit [our data]. The 40 corrections concentrated in pages written before the content contract existed — pages written under the contract audited clean — so the contract now precedes page one on every build. The propagated math error is why every derived figure is computed by script and used verbatim, never typed. The zombie claim produced the rule that a retired claim gains a build-failing regex the day it is retired.

The render-layer defects produced an output gate: pages are built and checked as HTML in CI, not just linted as source. And the fabricated regulation made refute-first the default audit posture on table-heavy and legal-adjacent pages — the auditor's job is to disprove the page from its own citations, not to confirm it feels right.

What is the generalizable rule?

Budget the audit as a fixed phase, because the gates will not catch what they were not built to catch. All 3 builds ran automated source gates from early on; all 3 audits still found real defects; each audit then hardened the gates for the next build. That loop — audit finds, gate encodes, next audit finds the next class — is the quality machinery inside the authority engine system, and its committed ledgers are the honest answer to "how do I know the AI didn't make things up," a question we think every buyer should ask — including of the people selling AI SEO.

Against our own interest: the audit is the expensive stage, it never gets cheaper with scale, and if your budget cannot fund it, the right move is fewer pages — the math is worked through in what a fact-audited page costs. A 9% correction rate on our own already-shipped content is the base rate we earned by checking [our data]; volume without verification just ships that rate unfound, on every surface an answer engine can quote.