A retrieval system answers from whatever it finds. If the best passage is weak, satirical or wrong, the answer will be too. Fix this with source ranking, a refusal path when evidence is thin, and tests for questions your documents do not cover.
What Google reported
In its 30 May 2024 post on AI Overviews, Google named several causes of bad answers: data voids, where little high quality content exists; satire, such as a rocks question answered from republished satirical content; sarcastic forum posts that led to glue on pizza advice; and occasional misreads of page language. It announced limits on satire and user generated content and restrictions on when overviews trigger.
Your internal assistant is much smaller than a search engine. The same pattern still applies.
Five checks for your system
1. Rank sources by trust. Not all documents are equal. Policies, official manuals and reviewed pages should outrank wikis, chat exports and old drafts. Store a trust tier as metadata and use it when retrieving.
2. Exclude what should never answer. Satire, jokes, brainstorming notes and outdated versions should not be in the index, or should be clearly tagged and filtered.
3. Create a refusal path. Set a minimum retrieval score. Below it, the assistant says it has no good source and points to a person or page. Google's own response included not triggering overviews for query types where they were not helpful. This is the small version of that.
4. Show the source. Display which document supported the answer, with a date. Readers can then judge a weak source immediately.
5. Test the voids. Write questions you know your documents do not answer, plus nonsense questions. The right behavior is a refusal, not an invented answer. See write evals before you fine tune for how to build such a set.
Watch user generated content
If your index includes tickets, chat or forum text, remember that people joke, guess and make mistakes there. Either leave it out of answers or label it as unverified.
A simple example
Suppose your support assistant is asked about a feature you retired last year. If the old help page is still indexed, it will answer with confidence. If the index has no page at all, a refusal path says so. If an internal chat export mentions the feature in a joking way, the assistant may quote the joke. Each case needs a different fix: archive the page, add the refusal, and exclude the chat export from answers.
Keep the index fresh
Old documents cause confident wrong answers just as well as bad ones. Add owners and review dates. Remove or archive pages that are superseded.
What this does not fix
Better retrieval lowers the rate of bad answers. It does not remove it. For high stakes questions, return approved text only or hand off to a person. For the failure when hidden text enters inputs, see hidden text and invisible characters in LLM inputs.
Quick check
What is a data void?
Google used the term for questions where little high quality content exists, so the system has weak material to answer from.
What should the assistant do with weak evidence?
Say it has no good source and direct the user to a person or an official page.
Sources
Last reviewed: 19 October 2026.
