Accuracy of AI: What a Major Study Found

Is AI always accurate? No. Not evenly, and not for every kind of task. A chat tool can rewrite a paragraph, categorise a list, or draft a reply with very little going wrong. Ask it to report a fact it was not directly given, or to summarise something longer than a few sentences, and the error rate climbs — not occasionally, but often enough that a careful study measuring it found real, counted gaps rather than a rare edge case.

What a major 2026 study actually found

The most rigorous recent measurement is not a vendor benchmark. A cross-market study coordinated by the European Broadcasting Union, with journalists from 22 public broadcasters in 18 countries reviewing thousands of answers to news questions, found that roughly 45% of the answers had at least one significant issue. About 20% had a major accuracy problem — an invented detail or outdated information stated as current. About 31% had a serious sourcing problem: a missing citation or one attributed to the wrong place. The tools involved varied in how often they got this wrong, but none of them were close to zero.

That study tested news questions specifically, which is a narrow and checkable kind of task — there was a real, published answer to compare against. Most work tasks are the same shape: there is a document, an email, a set of numbers, and a real answer sitting in it. The method below is how to use that fact.

The two different accuracy problems

Treat these as separate failures, because the fix for each is different. The first is being confidently wrong about something it was never actually told — guessing, in fluent prose that reads exactly like knowledge. What AI is actually bad at covers that failure mode in full, and a 2025 paper from OpenAI and Georgia Tech argues the behaviour persists because the way these models get graded rewards a confident guess over an honest “I do not know” — the guessing is not a flaw the training missed, it is what the training is actually optimising for. The second accuracy problem, and the one the EBU study measured most directly in its “invented detail” figure, is different: adding something that is not in the material you gave it, even when the source is sitting right there in the same conversation. That second failure is easier to catch, because you already hold the answer. You just have to actually compare.

A worked example: checking a summary against its source

Give the model a short, specific fact pattern — three or four plain sentences, nothing it has to infer. For example: “The team meeting moved from Tuesday to Thursday this week. The new room is 4B. Slides are due Wednesday at noon.” Ask for a one-paragraph summary a colleague could read in ten seconds.

Now read the summary against the three sentences, claim by claim. The addition to watch for is not usually a wrong number — it is a reason that was never given. A summary that says the meeting “moved because of a room conflict” has added a cause nobody stated. A summary that says slides are due “end of day” instead of “noon” has quietly loosened a deadline. Neither addition looks suspicious on its own. Both are invented, and both are the kind of small, plausible addition the EBU study’s “invented detail” figure is actually counting at scale.

Try a second version with a number instead of a reason: supply a one-line note that says a supplier invoice came to £1,240.50 with no mention of tax, and ask for a two-sentence summary for a finance colleague. Watch for the same pattern in a different shape — a summary that adds “including VAT” or rounds the figure to “around £1,200” has changed a fact that was stated plainly, not filled in a gap. The number itself did not have to be wrong for the summary to be wrong; it only had to say something the source did not.

Where it breaks

• —A summary adds a cause, motive or explanation that the source never stated, because a reason makes the sentence read more naturally than an unexplained fact.
• —A specific number, date or name gets smoothed toward something plausible-sounding when the source was vague or silent about it — what AI is actually bad at covers why precise-looking details are exactly the ones worth checking first.
• —A citation or attribution is missing or pointed at the wrong source entirely — the sourcing failure the EBU study measured in 31% of the answers it reviewed.
• —Fluent, calm phrasing carries the added detail exactly as confidently as the parts that were actually true, which is the underlying mechanism a widely cited survey of hallucination in large language models documents in detail.
• —A longer source makes this worse, not better — the more there is to compress, the more the model has to decide what counts as the gist, and “what counts as the gist” is exactly where an unstated assumption slips in as if it were a fact.

A summary that adds one small, accurate-sounding detail is harder to catch than one that gets everything wrong. The invented part is hidden inside several true ones.

Checks before you trust a summary

1. 01Keep the source text visible next to the summary, not just the summary on its own — you cannot compare a claim to a source you have already closed the tab on.
2. 02Check each claim in the summary against the source, sentence by sentence, treating any cause, number, date or name as the first thing to verify, not the last.
3. 03Ask directly: “What in your summary was not explicitly stated in the source?” It will often name its own additions when asked this way rather than to simply “double-check” — a more specific instruction gets a more specific answer, which is the same principle behind Anthropic’s prompt-engineering guidance.
4. 04For anything that will leave your desk — a client email, a report, a figure in a deck — run the check before it goes out, not after someone else flags it.

What to do Monday

Next time you ask a model to shorten or summarise something you already have the source for, run the claim-by-claim check above once, deliberately, before trusting the habit for the rest of the week. Deep research prompts covers the template for the harder version of this, where the source is several documents rather than three sentences, and ChatGPT cheat sheet is a useful list of what these tools are actually reliable at, so you are not checking everything equally hard. Generative AI use cases is worth a look if summarising is only one of several tasks you are weighing against each other, and AI answers questions covers the question before this one: which questions are even worth asking a model in the first place. For a profession where this exact checking discipline is the job, not a side skill, will AI replace auditors? covers what the data and the rules actually say.

Checking a claim against its source is a specific, teachable skill, not a vague instinct to “be careful” — and it is close to the skill behind the wage premium PwC has measured for workers with real AI skills: 62%, up from 57% the year before. Coursium teaches it directly, through ChatGPT Essentials. Stay ahead of AI by learning the tools on your phone.