A hallucination is a confident answer that isn't true or isn't supported by any source. In a live application, hallucinations are a security and trust problem as well as a quality one: a wrong policy answer can mislead a customer, and an invented fact stored by one agent can mislead every agent that reads it later. This lesson covers how teams ground answers in real sources, score how well an answer is supported, validate outputs before they're used, and decide when a person must check.
Grounding the answer
Grounding means tying the model's answer to trusted information instead of letting it rely on what it absorbed in training. Retrieval Augmented Generation (RAG) is the main technique: the application retrieves relevant passages and the model answers from them. RAG only helps when the right, correct information exists in the knowledge base and can be found.
Amazon Bedrock Knowledge Bases returns citations with each generated answer, linking parts of the response to the source chunks they came from. Citations don't stop a hallucination by themselves, but they make every claim checkable, by a user or by an automated check.
Confidence scoring and thresholds
A confidence score is a number that says how well an output is supported. Bedrock Guardrails' contextual grounding check gives each response a grounding score and a relevance score against the source and the user's question. A team sets a threshold between 0 and 0.99, and responses scoring below it are treated as hallucinations and blocked.
The threshold is a trade-off:
- Higher blocks more unsupported answers, but also more correct ones.
- Lower lets more answers through, including some weak ones.
AWS's agentic AI guidance suggests higher thresholds for safety-critical uses and lower ones for creative tasks, with thresholds tuned over time from logged results. The same guidance warns against one tempting shortcut: asking the model to rate its own confidence. A model sounds just as sure about invented facts as about real ones, so the score must come from an independent check.
Validating outputs
Several checks can sit between the model and the user:
- Automated Reasoning checks in Bedrock Guardrails translate a response into formal logic and test it against rules built from your policy documents. An INVALID result lists the policy rules the claim breaks. These checks only report findings; they never block, so the application must decide whether to serve, rewrite or withhold the answer.
- LLM-as-a-judge. A second model receives the source and the answer and assesses whether the answer is supported.
- Business checks. Facts such as prices, dates or account balances can be compared with the system of record before a reply is sent.
When a check fails, the application needs a fallback, such as adding a caveat, asking the user to clarify, or refusing to show the answer.
Human review
Human review gives the greatest confidence that an answer is correct, but it's slow and costly. AWS's generative AI guidance reserves it for the most critical responses and stresses that reviewers must be well trained. Automated checks handle the bulk of the traffic, and people handle the cases where a mistake could cause real harm.
On the exam
"Too many correct answers blocked" means lower the grounding threshold and retest. "Findings show INVALID but the answer still went out" means the application isn't acting on Automated Reasoning findings. "The model rates its own confidence" is the wrong answer; use an independent check. "Errors could harm people" points to trained human review.