What is Hallucination?
In LLMs, hallucination is when the model generates fluent, confident text that is factually incorrect, fabricated, or not supported by its training data.
Large language models predict the next token based on statistical patterns learned during training rather than retrieving verified facts. When the model lacks sufficient grounding for a query, it may still produce a plausible-sounding continuation by combining unrelated patterns.
This behavior arises because the training objective rewards coherence and fluency, not truthfulness. As a result, the model can invent details, citations, or events that never occurred while maintaining grammatical and stylistic consistency.
Techniques such as retrieval-augmented generation, fine-tuning with human feedback, and explicit source citation are commonly used to reduce the frequency and impact of hallucinations.
Example
When asked for the capital of Australia, a hallucinating model might confidently answer 'Sydney' and even add supporting details about its history, even though Canberra is the correct capital.
Why it matters
Hallucinations undermine trust in AI systems and can spread misinformation in high-stakes domains such as medicine, law, and education, making detection and mitigation a central challenge in deploying reliable LLMs today.
Frequently asked questions
They generate text by predicting likely word sequences rather than checking facts, so they can produce plausible but untrue statements when uncertain.
Related terms
Retrieval-Augmented Generation (RAG) is a technique that improves large language models by retrieving relevant external information before generating a response.
Prompt engineering is the practice of designing and refining text inputs (prompts) to guide AI models like large language models toward producing accurate, relevant, or creative outputs.
Fine-tuning is the process of taking a pre-trained AI model and continuing its training on a smaller, task-specific dataset to adapt it for a particular use case.
Temperature is a parameter in large language models that controls the randomness of generated text. Lower values produce more focused and deterministic outputs, while higher values increase creativity and variability.
Top-p sampling (nucleus sampling) is a text-generation technique that dynamically selects the smallest set of most likely next tokens whose combined probability exceeds a threshold p (e.g. 0.9), then samples from that set.