Is ChatGPT accurate? The number everyone quotes is not its score

The figure in circulation is that AI assistants get the news wrong 45 per cent of the time. It comes from real research, it is being quoted accurately as far as it goes, and it is not ChatGPT's score.

That study, published in October 2025, was coordinated by the European Broadcasting Union and led by the BBC. Twenty-two public service media organisations across 18 countries and 14 languages had journalists evaluate more than 3,000 responses from four assistants: ChatGPT, Copilot, Gemini and Perplexity. It found that 45 per cent of answers contained at least one significant issue, that 31 per cent had serious sourcing problems, and that 20 per cent contained major accuracy problems such as hallucinated details or outdated information. On a looser threshold, 81 per cent had some form of error.

The 45 per cent is the average across all four. The study published only one per-assistant number: Gemini, at 76 per cent, which the EBU described as more than double the others. ChatGPT, Copilot and Perplexity were not broken out. So anyone stating that ChatGPT is wrong 45 per cent of the time is attributing an aggregate — one that a single badly performing system pulled upward — to a specific product.

An earlier BBC study does allow that comparison. In February 2025, BBC journalists put 100 news questions to the same four assistants and scored the answers against seven criteria including accuracy, attribution and impartiality. Fifty-one per cent of answers had significant issues. On errors in how each assistant used BBC source material, the ranking was Gemini 34 per cent, Copilot 27 per cent, Perplexity 17 per cent, and ChatGPT 15 per cent.

Lowest of four. Which is worth stating precisely, because it is a narrower claim than it looks: that 15 per cent measures errors in the handling of BBC sources, not overall accuracy, and the study it comes from still found half of all answers significantly flawed.

The more useful finding is not the rate but where the failures cluster. The dominant problem in both studies is sourcing and attribution rather than fluency. Nineteen per cent of answers citing BBC content introduced factual errors — wrong statements, numbers, dates. Thirteen per cent of quotes attributed to BBC articles had been altered from the original or did not appear in the cited article at all.

That is the failure mode to understand. These systems do not produce obviously broken text you can spot by reading. They produce fluent, confident prose in which the attribution is wrong, the date has drifted, or the quotation has been smoothed into something nobody said. Nothing in the output signals which sentence is the unreliable one.

Peter Archer, the BBC's programme director for generative AI, said that despite some improvements there are still significant issues with these assistants. Jean Philip De Tender, the EBU's media director, described the problems as systemic, cross-border and multilingual.

So the practical answer depends on the task, and the studies above test the hardest version of it: open questions about current events, answered from whatever the system can retrieve. That is where sourcing breaks. Summarising a document you supply is a different task with a different failure rate, and neither study measures it.

If you take one operational rule from the research, take this one: the assistant's confidence carries no information about whether the attribution is real. Open the cited source. In roughly one case in eight, the quote will not be there.