1 in 4 AI Summaries Invents Something: How to Stop AI Hallucinating in Technical Documents
Have you ever pasted a technical document into a chatbot, got a clean summary back, and later found a number in it that appears nowhere in the original?
That invented number is the most common failure in AI summarization — and the fix is a workflow, not a cleverer prompt.
This post shows you how often AI invents details when it summarizes a technical document, the four-step workflow that catches the inventions, the five invented details to check first, and when a quick check is not enough. Figures are as of November 2026 unless noted.
How often does AI invent details in a summary?
Even the best models invent something. The question is how often — and on what kind of task.
Vectara's Hallucination Leaderboard is the most direct measurement we have. It hands each model more than 7,700 articles and asks for a summary using only the facts in the document, then scores the result with its own evaluation model, HHEM-2.3. In the October 22, 2026 update, 109 models ranged from a 1.8% hallucination rate at the top to 24.2% at the bottom. The median sat around 9.6% — about 1 in 10 summaries containing something the source never said.
Vectara ran these tests at temperature 0 where possible — the most deterministic setting. The inventions happened anyway. Even the best model adds unsupported content to about 1 in 55 summaries; the worst, about 1 in 4.
The task matters as much as the model. In a preregistered Stanford study (Magesh et al., Journal of Empirical Legal Studies, 2025), researchers ran 202 legal questions through Lexis+ AI and Westlaw's AI research tools — purpose-built systems that search a database before answering. Each tool still hallucinated in 17% to 33% of answers. A general-purpose model with no database access reached 43% in the same test, and an earlier study by the same group found GPT-4 hallucinating 58% of the time on questions about US federal court cases.
Now compare that with answering from memory. Stanford's AI Index 2026 compilation of a knowledge-question test found hallucination rates from 22% to 94% across 26 models when no source was provided. The pattern is consistent: a model with the document in front of it invents far less than a model answering from what it remembers.
It also helps to know there are two kinds of invention. An intrinsic hallucination contradicts the source — it flips a number or reverses a finding. An extrinsic hallucination adds something the source never said at all — a citation that does not exist, a specification the document never mentions. In technical documents, the extrinsic kind is the dangerous one, because a fake citation looks authoritative.
The 4-step workflow that stops invented details
Ground it, constrain it, extract before you write, then verify — in that order.
1. Give it the source — never ask from memory
Upload the file. Do not ask the model to summarize a document it has never seen.
Asking "summarize the ISO 9001:2015 requirements" from memory invites invention, because the model has to reconstruct the text from training. Uploading the document turns the job into reading, not recalling — and the numbers above show how large that difference is. OpenAI's own prompt engineering guide makes the same point: give the model reference text to constrain its response to resources you supply.
2. Constrain the prompt before you summarize
Write the rules into the prompt, so "I don't know" becomes a legal answer.
Use this block, pasted above the document:
Summarize ONLY from the document below. Use only facts stated in the document. Do not add outside knowledge, do not infer, and do not guess. If a figure, date, or claim is not in the document, write "NOT STATED." Copy every number exactly as written — do not round, convert units, or paraphrase values. For each key claim, quote the sentence or section it comes from.
Each line does a job. "NOT STATED" removes the pressure to answer — the pressure that makes models invent rather than admit a gap, which is why teaching a chatbot to say "I don't know" matters at all. The verbatim-numbers rule stops silent unit conversions and rounding. The quote requirement makes every claim checkable in seconds.
3. Extract first, then summarize
Summaries compress, and compression is where inventions slip in. Extraction lists; it does not compress.
Ask for a structured extraction first — every key claim with the section it appears in, every number with what it represents, every decision and deadline — then write the summary using only that extraction. We covered the same step in our guide to summarizing long PDFs, where it stops details from disappearing. Here it does the opposite job: it stops details from appearing out of nowhere, because a summary built from an inventory cannot contain a fact that was never inventoried.
4. Verify with the 5-minute check
Never forward a summary you have not checked. Five minutes catches most inventions.
- Count check: Ask how many sections the document has, and how many the summary covered. A mismatch means something was dropped — or added.
- Spot-check three numbers: Pick the three figures that matter most and find each one in the original. If one is wrong, check them all.
- Click every citation: A citation that does not open, or opens to a page that says something else, is invented. Fake citations are the most common invented detail in technical summaries.
- Run the "show me the sentence" challenge: "For each claim in your summary, show the exact sentence from the document that supports it." Unsupported claims surface immediately.
- Check units and versions: Conversions (°C to °F, mm to inches) and version numbers are where silent invention hides. Verify them against the original, not against the summary.
The 5 invented details to watch for in technical documents
Most invented details fall into five patterns. Check these first.
- Invented numbers: A figure that looks precise — "a 34% reduction" — but appears nowhere in the source. Precision is not evidence.
- Converted values: The model helpfully converts units and gets the math wrong, or converts a value the document never gave in that unit.
- Fake citations: A plausible author, journal, and year that do not exist — or a real paper cited for a claim it never makes.
- Wrong versions and dates: "As of the 2024 edition" when the document is the 2023 edition, or a deadline moved by a year.
- Merged claims: Two separate sections blended into one sentence that neither section supports. Common when a document has similar tables in different chapters.
When a 5-minute check is not enough
Use the quick check for internal reading. Escalate when the summary feeds a decision.
If the summary will be quoted to a client, filed, published, or used for a legal, medical, financial, or safety decision, add a second pass: run the same extraction through a different model and compare the numbers, or have a person check every flagged figure against the original. The Stanford result is the reason: even tools built for the job got hard questions wrong a third of the time.
One thing prompting cannot fix is temperature. Vectara ran its tests at temperature 0 — the most deterministic setting available — and still measured 1.8% to 24.2%. Consumer chatbots do not even expose a temperature control, so "turn down the temperature" is not an option in the chat window. The workflow is the control you actually have.
There is also a trade-off worth knowing: some models refuse rather than answer. On Vectara's board, one model answered only 62.7% of the time. A model that says "not stated" is more useful than one that invents smoothly — treat refusals as a feature, not a failure.
Key Takeaway
AI will keep inventing details in technical summaries. The best model on the public leaderboard still does it in about 1 in 55 summaries, and the worst in about 1 in 4. No prompt makes that zero — but a habit makes it visible before it costs you.
Ground the model, constrain the prompt, and verify the numbers — a summary is a draft until the source confirms it.
Sources
- Vectara Hallucination Leaderboard (GitHub, updated October 22, 2026) — github.com/vectara/hallucination-leaderboard
- Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025). "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools." Journal of Empirical Legal Studies, 22:216–242 — doi.org/10.1111/jels.12413
- OpenAI Prompt Engineering guide — platform.openai.com/docs/guides/prompt-engineering
- Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024). "Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models." Journal of Legal Analysis.
- Stanford HAI, AI Index Report 2026, Chapter 3.

Comments
Post a Comment