AI Fakes: 146,900 Fake Citations Found in Science

A significant number of fake citations, totaling 146,900, have been discovered in scientific papers, according to a study by researchers from Cornell and UCLA. These AI-generated hallucinations were found across four major research databases: arXiv, bioRxiv, SSRN, and PubMed Central. The study analyzed 111 million references from 2.5 million papers, identifying an alarming rise in non-existent citations following the widespread adoption of large language models.

This proliferation of fabricated references, which began before the advent of chatbots but has accelerated since, undermines the integrity of the scholarly record. The fake citations are dispersed across many papers, indicating a widespread issue where researchers may be relying on AI-generated citations without proper verification.

In response to the growing problem of AI-generated content and its potential to dilute the scientific corpus, organizations like arXiv are taking action. arXiv announced it will ban authors who submit work containing hallucinated citations or unchecked AI content, aiming to preserve the quality and reliability of scientific research.