Science's 'credit score' dream falls short

A significant initiative, dubbed SCORE and backed by DARPA, sought to develop artificial intelligence capable of predicting the trustworthiness of scientific publications. This ambitious project, initiated in 2019, involved 865 researchers analyzing thousands of studies. The goal was to establish a 'credit score' for scientific findings, but current AI capabilities have proven insufficient for this complex task.

During the project, researchers replicated 164 studies and found that only about half produced consistent results. Further analysis revealed that variations in methodology and even coding errors could lead to conflicting outcomes. While AI showed some predictive ability, it was not accurate enough for independent use, indicating that a 'scientific credit score' remains an elusive goal for now. Despite this, the SCORE effort has provided valuable insights into improving the scientific process, with ongoing efforts to enhance data sharing and the publication of replication studies.