Why AI answers need checking
An AI model writes the most likely next words, and a wrong figure is often as likely as a right one. It states both with the same confidence, so the tone of an answer tells you nothing about whether it is true. The guide to hallucinations explains why this happens and how to make it rarer.
Checking does not mean redoing the work. Most of an answer is reasoning, structure and wording you can judge by reading. The part that needs checking is narrow: the specific facts. That is why the first step is to find them.
1. List what can be checked
Go through the answer sentence by sentence and mark every claim that could be true or false in the world, not just in the answer. These are the kinds that matter:
- Numbers. Statistics, percentages, prices, sizes, growth rates.
- Named sources. “A 2023 study by…”, a report, a survey, an organization.
- Quotes. Anything in quotation marks, especially attributed to a famous person.
- Dates and people. When something happened, who holds which job, who said what.
- Laws, rules and court cases. Section numbers, case names, what a regulation requires.
- Links and references. Web addresses, paper titles, DOIs.
- Superlatives and vague authority. “The largest”, “the first”, “experts agree”, “studies show”.
The Fact Check List does this step for you: paste the answer and it lists every sentence with a checkable claim, labelled by kind. It runs in your browser and never decides whether a claim is true; that part stays yours.
2. Check the riskiest first
Not every claim is equally likely to be wrong. Start where models fail most and where an error costs most.
| Kind of claim | Why it goes wrong | How to check it |
|---|---|---|
| Links, paper titles, case names | Models produce references that look right and do not exist, unless they used a search tool, and sometimes even then. | Open it. Search the exact title in quotes. For a case, search a court database. |
| Quotes | Famous people get credited with lines they never said, and real quotes get reworded. | Search the exact wording in quotes and find it in the original speech, book or interview. |
| Statistics with a named source | The source is often real and the number is not in it, or comes from a different year or country. | Find the figure in the source itself, not in an article about it. |
| Dates, titles, who did what | Out of date, or mixed up with a similar person or event. | One reliable, current source: the organization’s own page, a news report, a register. |
| “Studies show”, “experts say” | No source to check at all, which is a warning in itself. | Ask which study. If there is none, cut the claim or soften it. |
3. Go to the original source
Check a claim against where it comes from, not against another summary of it. A news article about a survey can misreport it, and a second AI answer can repeat the first one’s mistake. Professional fact-checkers do this by leaving the page: they look at what other reliable sources say about a claim and its source, instead of reading one page more closely.
Prefer, in order: the original document (the study, the law, the company’s own filing), then an official page that cites it, then a reputable report that links to it. If you cannot find the original in a few minutes, treat the claim as unconfirmed.
4. Open every link and quote
Never repeat a link you have not opened. A web address in an AI answer can be well formed and lead nowhere, or lead to a page that says something else. Open it, find the sentence that supports the claim, and only then keep it.
For quotes, search the exact words inside quotation marks. If the only results are quote collections with no original source, the attribution is doubtful.
Getting the AI to help
An AI can help you check, as long as you remember it may repeat its own mistake. Two prompts are worth keeping:
List every factual claim in your answer above as a numbered list. For each one, say where it comes from, and mark any you are not sure of with "unverified". Do not add new claims.
With web search turned on, ask for a source link next to each claim, then open the links yourself. Search makes answers better grounded, but a page can be wrong and a summary can misread it. For a document you supplied, the claim-checking template compares a draft against your sources and quotes the passage behind each claim.
When you can skip it
Not every answer needs this. Brainstorming, rewording your own text, a formula you will test anyway, or a summary of a document you will read: the check is built in. Fact-check when the answer will leave your hands, inform a decision, or be read by someone who will trust it because it came from you.
Key takeaways
- An AI answer sounds equally sure when it is wrong, so tone tells you nothing.
- Check the specific facts, not the whole answer: numbers, sources, quotes, dates, people, laws and links.
- Start with links, references and quotes, which models invent most often.
- Check against the original source, not another summary or a second AI answer.
- Never repeat a link you have not opened.