AI slop just failed a 10-week exam. The fact-checking charity Full Fact asked the same 22 questions every day to five AI chatbots — Claude, ChatGPT, Gemini, DeepSeek and Grok — and caught them confidently giving wrong answers: false claims, outdated figures, and different answers to the same question on different days.
AI hallucinations are fluent, confident answers that are factually wrong — invented details, stale facts, or botched reasoning delivered as if true. They are dangerous precisely because they sound right. This page is built around a real study that measured them, plus a quiz that trains your eye to catch them.
The ex-prime minister. A chatbot asserted Sir Keir Starmer was still UK prime minister as late as September 2026 — weeks after he had resigned. Not hedged, not qualified: stated as current fact.
The confident maths. On August 4, Gemini reported 12,210 UK small-boat migrant arrivals for the first half of 2026. The arithmetic was flawless — 14,526 (total as of August 1) minus 2,316 (July arrivals). But it forgot to exclude August 1 itself, so the "first half" figure silently covered January to June plus one extra day. Right sum, wrong period, zero doubt expressed.
Rogue-Agent Defense — Ingenious AI's AI-reliability platform — is built for exactly this failure mode: autonomous systems that act on confident-but-wrong outputs. Get early access on WhatsApp.
Five questions drawn from the study. Click an answer; your score appears at the end.
1. Full Fact asked the same 22 questions daily for 10 weeks to 5 chatbots. What did it find?
2. A chatbot said Sir Keir Starmer was still UK prime minister in September 2026, weeks after he resigned. What hallucination pattern is this?
3. Gemini's small-boats figure: 14,526 − 2,316 = 12,210. The arithmetic was right — so what went wrong?
4. The Full Fact report ALSO found that the chatbots…
5. You need a figure from a chatbot that you will publish. The safest move?
From mid-July 2026, the fact-checking charity Full Fact ran a 10-week study asking the same 22 questions every day to five AI chatbots — Claude, ChatGPT, Gemini, DeepSeek and Grok — testing accuracy, consistency and timeliness of their answers.
AI hallucinations are confident, fluent answers from a chatbot that are factually wrong — false claims, outdated figures, or invented details presented as if true. The Full Fact study found chatbots delivered wrong answers with total confidence rather than hedging or refusing.
No — and this matters. The Full Fact report said the chatbots often gave correct answers and resisted well-established misinformation. The failures were specific: they calculated figures themselves instead of citing official statistics, or presented outdated figures as current.
On August 4, 2026, Gemini reported 12,210 UK small-boat migrant arrivals for the first half of 2026. Its arithmetic was correct — it subtracted 2,316 (July arrivals) from 14,526 (total as of August 1) — but it forgot to exclude August 1 itself, so the figure wrongly covered January to June plus one extra day.
Ask the chatbot to cite the official statistic and its date, then verify both independently. Treat confident tone as a style choice, not evidence. For anything you will publish, repeat or act on, cross-check against the primary source — the study showed confidence is the worst possible proxy for correctness.
Sources: The Independent's report on the Full Fact study (October 14, 2026); Full Fact (fact-checking charity, 10-week study from mid-July 2026). Study scope: UK-focused questions; results may differ by topic and region.