Reliability & risk
Hallucination is the central trust problem with AI. Tying answers to your own documents, and showing the source, is how serious systems keep it rare and catchable.
In plain terms
A hallucination is when an AI model states something false with full confidence, inventing a fact, source, or number that looks completely plausible.
A hallucination is a fluent answer that is not true. The system fills a gap because it is built to produce a likely next sentence, not to stay silent. It can invent a policy clause, a court case, a product spec, or a number. The writing will not look uncertain.
You reduce it by giving the system the document, showing the source, and telling it to say when the document does not contain the answer. You do not remove it by asking the model to be careful. For anything a customer, a patient, or a regulator could rely on, a person checks before it goes out. Treat a confident unsourced answer as untrusted.
Think of it this way: Hallucination is an AI confidently stating the wrong flight time. The tone, format, and surrounding context look completely reliable. The specific fact is invented.
A legal tech tool, without grounding, cites a case that does not exist. The lawyer who submits the brief faces a professional conduct review. One hallucination with no guardrail became a serious liability.
A salesperson asks the internal bot for the warranty on a product the company does not sell. The bot describes a two-year warranty in a confident paragraph. There is no source because there is no product. The corrected bot says it cannot find that product in the catalog and stops.
Treat this as a known failure, not a feature. For any answer that could cause harm, tie it to your own documents, show the source, and have a person check it before it goes out. Do not treat a made-up answer as something you have to live with. Tying answers to your own documents, and testing the system against real examples, cuts this sharply. The question is whether that work is in the plan.
RaftLabs treats this as part of the build: a source on the answer, a test set, and a record of what the system did. The launch is the start of that work, not the end. The related work on our side is RAG development.
This sits with the other reliability & risk terms on the glossary. Why a confident answer can still be wrong, and how you catch it. Worth reading next: Guardrails, Evaluation (evals), and Model Drift.
Work with us
Tell us what's not working in your business. We'll find the real problem and tell you exactly what it would take to fix it.