Home / Health / Study: Chatbots Offer No Better Health Answers
Study: Chatbots Offer No Better Health Answers
9 Feb
Summary
- AI chatbots provide no better health decisions than internet searches.
- Large language models identified conditions in 94.9% of cases.
- Human participants using AI showed no improvement over traditional tools.
A recent study published in Nature Medicine indicates that artificial intelligence chatbots do not assist patients in making superior health decisions compared to conventional methods such as internet searches. Researchers from the University of Oxford's Internet Institute highlighted the growing reliance on AI for health advice without evidence of its safety or efficacy.
When tested on 10 medical scenarios, three leading AI models accurately identified conditions in 94.9% of cases. However, they suggested the correct course of action in only 56.3% of instances. The study involved 1,298 participants in Britain who used either AI, internet searches, or the NHS website to investigate symptoms.
These participants, utilizing AI, identified relevant conditions in less than 34.5% of cases and suggested the correct action in under 44.2%. This performance was no better than control groups using traditional tools. Experts noted a significant gap between AI's capabilities and its practical application by users, suggesting that AI's knowledge does not always translate effectively during human interaction.
