New OpenAI study shows why AI can get it wrong and how to fix it

A new OpenAI study explains why ChatGPT and other large language models sometimes make up facts. It turns out that these errors are mathematically inevitable because of the way AI forms answers - it predicts words one by one.

This is reported by ScienceAlert.

Errors accumulate when constructing complex sentences, and the probability of "hallucinations" increases for facts that the model rarely saw during training. Even modern models can give several different incorrect birth dates for the same person, so they give wrong answers in seemingly simple facts.

The problem is exacerbated by the fact that modern methods of evaluating AI encourage "guessing" instead of expressing uncertainty. If the model says "I don't know", the user does not like it. Therefore, the system answers confidently, but sometimes incorrectly.

How to fix "hallucinations" of language model

OpenAI says that a language model can be trained to assess its own confidence and respond only when it exceeds a certain threshold. This would reduce "hallucinations", but then users would not always receive answers.

Another barrier is computational cost. Models that take uncertainty into account require significantly more resources. For critical areas such as medicine or finance, this is justified, but for mass consumer applications it is economically unprofitable.

Recall that Sam Altman announced new rules for using ChatGPT, which apply to underage users. The company claims that it puts the safety of teenagers above privacy or freedom of interaction.