AI Safety: Accuracy and misinformation

An explanation of how to spot and mitigate inaccuracies in generative AI outputs.

Generative Artificial Intelligence (AI) outputs are not always accurate or reliable. This is a problem when false information is reproduced and cited in academic research or public communication. The result is public misinformation, disinformation, or even fake news.

To understand where the inaccuracies come from, it is important to highlight that Generative AI cannot reason or ‘understand’ truth. This is because Generative AI is based on a probabilistic model, meaning they generate outputs based on likelihoods, and can produce different responses to the same input. Because these models generate outputs by predicting statistically likely words rather than checking factual correctness, they can produce plausible-sounding but inaccurate information.

False information can make its way into the model in various ways:

  • Training data could be out of date or incorrect from the outset.
  • The AI could be programmed to always provide answers regardless of whether it finds the information the prompt asks for.

Laws and guidelines

It is important to engage with relevant questions and guidelines regarding AI accuracy. Accuracy and related cybersecurity concerns are beginning to show up in AI regulation and legislation, like the EU Artificial Intelligence Act.

Article 15: Accuracy, Robustness and Cybersecurity | EU Artificial Intelligence Act

AI Ethics Learning Toolkit - Can we trust AI? | Duke Center for Teaching and Learning 

Why is accuracy important?

Generative AI inaccuracies can cause issues in various subject areas. This list of risk scenarios is not exhaustive, but shows a snapshot of how the problem could affect your studies and work at the university:

  • The design of science experiments may lack safety considerations or even be faulty to the point where it causes physical harm to researchers.
  • Generative AI systems regularly fabricate references and citations, making them look realistic although the actual text does not exist.
  • AI generated code often introduces security vulnerabilities and lacks data for the latest digital safety practice. Implementing insecure code also affects overall cybersecurity in the digital world.

What are hallucinations?

When an AI tool generates content which is fabricated or inaccurate, it is often referred to as ‘hallucination’. Although many AI systems are coded to give accurate information, this may, for example, contradict its code to always provide an answer rather than state it does not have one.

The term ‘hallucinations’ is much debated as it falsely suggests human activity, obscures the technical reason for inaccurate outputs and shifts responsibility away from AI developers. In summary, it anthropomorphises (humanises) the AI and lacks technical accuracy.

How do you ensure accuracy when using AI?

‘Human oversight’ is the core ingredient to ensure the accuracy of generative AI outputs. While spreading misinformation accidentally is easily done, it is your responsibility to apply due diligence to the AI outputs you share and keep safety at the forefront of your digital activities. 

Guidance on How We Can Counteract Generative AI’s Hallucinations? | Digital Data Design Institute

  • Avoid using generative AI like a search engine – that is not their purpose or strength.
  • Verify and critically assess sources and facts you receive from a generative AI system. Use reputable sources to do so (see guides below for how to evaluate sources for credibility).

How to evaluate website content | University of Edinburgh Library

What are reliable sources? | BBC Bitesize

  • Be specific in your prompts and query suspicious AI outputs in your follow-up prompts.
  • Reference your use of AI in your research and writing.
  • Reflect on the impact the information you aim to share can have on society and politics (for example fake news).
  • Adjust the temperature of your generative AI tool. This means, control in the system settings how imaginative (higher temperature) or focused and factual (lower temperature) outputs should be.

All major AI models risk encouraging dangerous science experiments | New Scientist (no date). Available at: https://www.newscientist.com/article/2511098-all-major-ai-models-risk-encouraging-dangerous-science-experiments/ (Accessed: 2 March 2026).

Generative AI content: The need for accuracy (no date) Charity Digital. Available at: https://charitydigital.org.uk/topics/generative-ai-content-the-need-for-accuracy-11306 (Accessed: 2 March 2026).

Islam, N. (2025) ‘The Fabrication Problem: How AI Models Generate Fake Citations, URLs, and References’, Medium, 12 June. Available at: https://medium.com/@nomannayeem/the-fabrication-problem-how-ai-models-generate-fake-citations-urls-and-references-55c052299936 (Accessed: 2 March 2026).

The Concern Around Saying AI ‘Hallucinates’ (no date) United Nations University. Available at: https://unu.edu/article/concern-around-saying-ai-hallucinates (Accessed: 2 March 2026).

‘When AI Gets It Wrong: Addressing AI Hallucinations and Bias’ (no date) MIT Sloan Teaching & Learning Technologies. Available at: https://mitsloanedtech.mit.edu/ai/basics/addressing-ai-hallucinations-and-bias/ (Accessed: 2 March 2026).

Why language models hallucinate (2026). Available at: https://openai.com/index/why-language-models-hallucinate/ (Accessed: 2 March 2026).

Zhou, Y. et al. (2026) ‘Benchmarking large language models on safety risks in scientific laboratories’, Nature Machine Intelligence, 8(1), pp. 20–31. Available at: https://doi.org/10.1038/s42256-025-01152-1. 


© Ricarda Fillhardt, University of Edinburgh, 2026, CC BY-SA 4.0, unless otherwise indicated. 

This article was published on