Artificial intelligence regularly makes large and small errors in news texts, so it cannot be a reliable source of news. This conclusion was reached by researchers from the European Broadcasting Union (EBU), who analyzed the responses of the most popular AI assistants in different languages.
The results of the study are published on the website of the union.
The EBU involved 22 public broadcasters from 18 countries and analyzed 3,000 news responses from ChatGPT (OpenAI), Copilot (Microsoft), Gemini (Google) and Perplexity. Accuracy, reliability of sources, separation of facts from opinions and context were assessed.
The results were disappointing: 45% of all responses contained serious errors, and 81% contained minor inaccuracies. Most often, the problems concerned sources: in 31% of cases, data was missing, distorted or incorrectly attributed.
In second place are accuracy errors (30%), which include fabricated details and outdated information. For example, ChatGPT in the test stated that Pope Francis is still the current pontiff, although the simulation indicated that he had already died and was succeeded by Pope Leo XIV. Copilot referred to a 2006 BBC news story when answering a question about modern research on bird flu.
Gemini performed the worst - problems were recorded in 76% of its answers, twice as many as in competitors. For comparison: Copilot - 37%, ChatGPT - 36%, Perplexity - 30%.
Assistants were most often mistaken in topics that change quickly, require complex context or a clear line between facts and judgments. Almost half of the models could not cope with the question: "Is Trump starting a trade war?".
Despite this, AI is becoming an increasingly popular tool for obtaining information. According to the Reuters Institute and the University of Oxford, the number of people using AI for news has doubled in a year, most notably in Argentina, the US and among young people aged 18-24.
Researchers warn that if users trust the reliability of an AI that is wrong, this creates a danger - because such answers distract attention from trusted media.
"Artificial intelligence assistants are still not a reliable way to get news," EBU experts concluded.
According to them, the improvement in accuracy compared to the previous BBC study is currently insignificant.
As Ukrainian News Agency earlier reported, Italian media publishers have called for an investigation into Google's AI Overviews, claiming that the automatic search results service is harming their traffic and threatening the survival of publications.
Who we are: About us, Contacts. How we write news and our principles: Editorial code. We did our best. If you found this valuable – please support us.
To request a correction, please send an email.