Are AI medical tools more likely to provide incorrect advice?

Yes, AI tools are more likely to provide incorrect medical advice when the misinformation comes from what the software considers an authoritative source.

What type of source is more likely to trick AI with medical misinformation?

AI software is more often tricked by mistakes in realistic-looking doctors' discharge notes than by mistakes in social media conversations.

What is needed to improve AI in healthcare?

AI needs built-in safeguards that check medical claims before they are presented as fact to strengthen systems before they are embedded in care.

Home / Science / AI Medical Advice: Source Matters for Accuracy

AI Medical Advice: Source Matters for Accuracy

10 Feb

•

Summary

AI tools more often fail with medical misinformation from authoritative sources.
Discharge notes, not social media, more likely to trick AI.
AI believed false hospital notes nearly 47% of time.

AI Medical Advice: Source Matters for Accuracy

A recent study indicates that artificial intelligence tools are more likely to provide erroneous medical advice when the misinformation stems from a source the AI deems authoritative.

Testing revealed that AI software was more readily misled by errors in simulated doctors' discharge notes than by inaccuracies found in social media discussions. This suggests that current AI systems may prioritize confident presentation over factual accuracy in medical contexts.

The research exposed AI to various content types, including fabricated hospital discharge summaries and common health myths. Findings showed that AI tools accepted false information from authoritative-looking sources nearly 47% of the time, a significant increase from the 32% overall rate of belief in misinformation.