AI Hallucinations — When AI Confidently Makes Things Up
Status: 🟩 COMPLETE 🟦 LIVING Tags: hallucinations, accuracy, AI-reliability, confabulation, reliability
What it is
An AI hallucination is when an AI model generates information that sounds plausible and confident but is factually wrong — or completely invented. The AI isn’t lying (it has no intent); it’s producing what is statistically likely given its training, without any awareness that the output is incorrect.
The term “hallucination” comes from psychology — the AI is producing something that isn’t there in reality. Some prefer the term confabulation (filling gaps with plausible-sounding fiction) because “hallucination” implies sensory perception the AI doesn’t have.
Hallucinations are one of the most important limitations of current AI systems, and understanding them is essential for anyone using AI for anything important.
Common types of hallucinations
| Type | Example |
|---|---|
| Fake citations | ”According to Smith et al. (2021) in the Journal of Medicine…” — the paper doesn’t exist |
| Fake statistics | ”87% of Australians…” — plausible number that was never measured |
| Incorrect facts | Wrong dates, wrong names, wrong locations — stated with complete confidence |
| Invented events | A news story that never happened, described in convincing detail |
| Wrong expertise | Technically confident medical, legal, or scientific claims that are incorrect |
| Misattributed quotes | Famous quotes attributed to the wrong person (or quotes that no one actually said) |
| Outdated information presented as current | Facts that were true in the training data but changed since |
| Fake product details | Invented specifications, pricing, or features for real products |
Why hallucinations happen (plain English)
Language models work by predicting what text should come next, given what came before. They don’t have access to external truth; they generate based on patterns learned from training data.
Think of it like a very sophisticated autocomplete:
- Your phone’s autocomplete suggests words based on patterns from millions of texts
- It doesn’t “know” if a word is correct in context; it just knows what tends to follow what
- LLMs do the same thing, but at the scale of entire documents — they generate what “sounds like” a correct response, based on patterns
When an AI says something confidently wrong, it’s because:
- The training data included similar-sounding but incorrect information
- The pattern “a paper about X by someone named Y in year Z” is common → it generates that pattern without checking if that specific paper exists
- It has no mechanism to distinguish “things I know to be true” from “things that sound plausible”
How common are hallucinations?
Hallucination rates vary significantly by model, task, and how you measure:
- For simple factual questions where information was well-represented in training data: hallucination rates can be low (under 5%)
- For obscure topics, recent events (after training cutoff), or niche details: rates are much higher
- For citations and specific claims with verifiable details: historically very high; improving with grounding (RAG)
- For reasoning tasks: lower, especially in models with chain-of-thought
Frontier models (GPT-4o, Claude 3.5, Gemini 1.5 Pro) hallucinate significantly less than earlier models, but they still hallucinate. This is a major area of active research and improvement.
Which tasks have the highest hallucination risk
🔴 High risk:
- Academic citations (paper titles, author names, journal names, page numbers)
- Specific statistics and numbers from specific studies
- Legal case citations and statute numbers
- Medical research details
- Historical dates and exact sequences of events
- Product specifications and pricing
- People’s biographical details (especially lesser-known people)
🟡 Medium risk:
- General factual claims
- Technical explanations
- Event descriptions
- Company and organisation details
🟢 Lower risk:
- Reasoning and analysis
- Summaries of provided text (when the source text is in the prompt)
- Creative writing (where “truth” doesn’t apply)
- Code generation (verifiable by running it)
- Mathematical calculations (verifiable)
How to reduce hallucination risk
1. Verify everything important
Never use AI-generated specific facts, numbers, or citations in any important context without verifying them against primary sources. This is the most important rule.
2. Use RAG (Retrieval-Augmented Generation)
See rag. When AI is given actual source documents to reference, and is told to answer only from those documents, hallucination rates drop dramatically.
3. Use deep research mode for factual research
See deep-research-mode. Tools like ChatGPT Deep Research and Perplexity actively fetch and cite real web sources, reducing (though not eliminating) hallucination.
4. Ask for sources, then verify them
Ask the AI to cite its sources. Then verify that those sources exist and say what the AI claims. This catches citation hallucinations.
5. Use models with knowledge cutoff awareness
Better models now acknowledge when a topic might be outside their training data. If an AI says “I’m not certain about this,” take that warning seriously.
6. Cross-check with other sources
For any critical information, use multiple sources (including non-AI sources).
7. Prefer AI for tasks less prone to hallucination
For creative work, reasoning, analysis of provided text, and brainstorming — hallucination risk is much lower. For “what is the exact value of X?” — verify.
Hallucinations in specific contexts
Legal: very high stakes
The most famous hallucination cases have been legal. In 2023, a US lawyer submitted an AI-generated brief citing multiple cases that didn’t exist. The cases sounded real — “Santiago v. United Airlines” with plausible case numbers and quotes. The lawyer was sanctioned.
In Australia: Australian courts and the Law Council have issued guidance that lawyers must verify AI-generated content and remain personally responsible for court filings.
Medical: potentially dangerous
AI can generate plausible-sounding medical advice that is incorrect or dangerously incomplete. Never rely on AI for medical diagnosis, dosing, or treatment decisions without medical professional verification.
Journalism: growing concern
AI-generated misinformation and hallucinated quotes attributed to real people present serious challenges for journalism. Many publishers now require disclosure of AI-generated content and fact-checking workflows.
Education: citation problem
Students submitting AI-written essays with fake citations are being caught because the cited papers don’t exist. Verify every citation.
Grounding: the main solution direction
The most promising technical approach to reducing hallucinations is grounding — connecting the AI to real, verified information sources rather than relying purely on training data.
- RAG: Give the AI real documents to read and quote from
- Tool use: Let the AI search the web, run code, or query databases to get real answers
- Constitutional AI / RLHF: Train models to be more calibrated — to say “I don’t know” when they don’t know
None of these eliminate hallucinations entirely, but they significantly reduce rates. The field is improving rapidly.
Gotchas
- Confident tone is not a reliability signal. AI models typically present hallucinated information with the same confidence as correct information. The confidence of the statement tells you nothing about its accuracy.
- “Sounds right” is not “is right.” Hallucinations are plausible — they follow the patterns of real information. A fake citation has realistic author names, journal names, and years.
- Hallucinations can be hard to detect without domain knowledge. A hallucinated medical claim may be hard to spot unless you’re a doctor. This is why AI is particularly risky in expert domains.
- Models don’t know they’re hallucinating. An AI cannot reliably tell you whether a specific claim it just made is accurate. It has no access to an internal “truth meter.”
- Corrections can introduce new hallucinations. Sometimes correcting an AI on one point causes it to change other (correct) information in the process. Always re-verify the full response after corrections.
See also
- rag — the main technical mitigation for hallucinations
- deep-research-mode — AI research that fetches real sources
- prompt-engineering — prompting techniques that reduce hallucination
- prompt-injection — a related AI reliability/security issue
- ai-safety-primer — broader AI reliability and safety issues
Sources
- Maynez et al., “On Faithfulness and Factuality in Abstractive Summarization” (ACL 2020) — foundational hallucination research
- Bang et al., “A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity” (2023)
- Rawte et al., “A Survey of Hallucination in ‘Large Foundation Models’” (2023)
- Mata v. Avianca (SDNY 2023) — the landmark legal hallucination case
- NIST AI Risk Management Framework (2023) — hallucination risk documentation
- Law Council of Australia — AI guidance (2024)