Hallucination in LLMs

Hallucination in LLMs is when a model generates text that is fluent and confident but factually incorrect, fabricated, or unsupported by the input context or the model's training data.

What It Really Means

LLMs are probabilistic text generators. They predict the most likely next token given the preceding context. They do not have a concept of truth — they have a concept of plausibility. When the model generates "The Eiffel Tower was built in 1887" instead of 1889, it is not lying. It is generating a plausible-sounding completion that happens to be wrong.

Hallucination is not a bug to be fixed — it is an inherent property of how language models work. The same mechanism that allows an LLM to generate creative fiction, write code it has never seen, and answer novel questions also allows it to fabricate citations, invent API endpoints, and confidently state incorrect facts.

This is the central reliability challenge in AI engineering. Every production LLM application must have a hallucination mitigation strategy. The approaches range from grounding responses in retrieved documents (RAG) to post-generation validation (AI guardrails) to architectural choices that reduce the opportunity for hallucination.

How It Works in Practice

Types of Hallucination

Intrinsic Hallucination: The model contradicts information in its input.

• Context says: "Revenue grew 15% in Q3"
• Model says: "Revenue grew 25% in Q3"
• The model distorts provided facts

Extrinsic Hallucination: The model generates information not present in its input or verifiable from training data.

• Model says: "According to Smith et al. (2023), the algorithm achieves 99.2% accuracy"
• No such paper exists. The citation is fabricated.

Semantic Hallucination: The model generates text that is syntactically correct but semantically nonsensical.

• "The patient's blood pressure was 120/80 degrees Celsius"
• Mixing up units and medical measurements

Why It Happens

1. Training data gaps: The model has insufficient or contradictory information about a topic
2. Pattern completion: The model follows a plausible pattern rather than recalling a specific fact
3. Frequency bias: Common patterns override rare but correct information
4. Context window limitations: Relevant information is lost in long contexts ("lost in the middle")
5. Decoding randomness: Higher temperature increases diversity but also hallucination risk

Real-World Impact

• Legal: Lawyers cited fake cases generated by ChatGPT in court filings (Mata v. Avianca, 2023)
• Medical: AI chatbots provided incorrect medical dosage information
• Code: Generated code references non-existent libraries or API methods
• Search: AI-generated summaries present fabricated statistics as facts

Implementation

Detection: Checking for Hallucination

Mitigation: RAG with Citation Verification

Mitigation Strategies Summary

1. RAG: Ground responses in retrieved documents
2. Temperature reduction: Lower temperature = less creative = less hallucination
3. Citation requirements: Force the model to cite sources
4. Self-consistency: Generate multiple responses and check for agreement
5. Guardrails: Post-generation validation and filtering
6. Constrained generation: Limit output to known valid options
7. Retrieval verification: Cross-check generated facts against retrieved documents

Trade-offs

Strict Anti-Hallucination Measures

• Higher accuracy and reliability
• More "I don't know" responses (lower coverage)
• Higher latency (verification takes time)
• Higher cost (additional LLM calls for checking)

Lenient Measures

• Higher coverage — model attempts to answer more questions
• Risk of confident but incorrect responses
• Lower latency and cost
• May be acceptable for low-stakes applications

Advantages of Addressing Hallucination

• Builds user trust in AI-generated content
• Reduces liability in regulated industries
• Enables AI deployment in high-stakes domains (healthcare, legal, finance)

Disadvantages of Over-Correction

• Models become overly cautious and refuse to answer valid questions
• Verification overhead can negate the speed benefits of AI
• May require domain experts to calibrate acceptable thresholds

Common Misconceptions

• "RAG eliminates hallucination" — RAG reduces hallucination by grounding responses in sources, but the model can still misinterpret context, mix up facts from different chunks, or generate plausible but unsupported inferences from the source material.

• "Hallucination will be solved in the next model version" — Hallucination is inherent to probabilistic text generation. Newer models hallucinate less but still hallucinate. There is no indication of a complete solution on the horizon.

• "Confident responses are more likely to be correct" — LLMs can be maximally confident while completely wrong. Confidence is about probability of token sequences, not factual accuracy. There is no reliable internal signal for truthfulness.

• "You can detect all hallucinations automatically" — Automated detection catches many hallucinations but not all. Subtle factual errors, plausible-sounding fabrications, and domain-specific inaccuracies often require human expert review.

• "Low temperature prevents hallucination" — Low temperature makes the model choose the most probable tokens, but the most probable completion can still be factually wrong. Temperature affects style, not factual accuracy.

How This Appears in Interviews

Hallucination is a critical topic in AI engineering interviews:

• "How would you build a medical information system using LLMs?" — the hallucination risk makes this a challenging design problem. Discuss RAG, multi-stage verification, human review, and guardrails. See our guides on AI engineering.
• "Your customer reports that the chatbot is making up product features. How do you investigate and fix this?" — discuss logging, retrieval quality analysis, faithfulness evaluation, and the balance between coverage and accuracy.
• "How do you measure hallucination rate in production?" — discuss automated faithfulness checks, sampling-based human evaluation, and continuous monitoring.

Related Concepts

• RAG — Primary mitigation strategy for hallucination
• AI Guardrails — Post-generation safety checks
• Prompt Engineering — Prompt design affects hallucination rate
• Fine-Tuning vs RAG — Each approach has different hallucination profiles
• Semantic Search — Better retrieval reduces hallucination in RAG
• Algoroq Pricing — Practice AI reliability interview questions