AI Medical Advice: Half Problematic, Study Finds

Artificial intelligence chatbots, such as ChatGPT and Grok, are frequently providing inaccurate and incomplete medical information, a recent study has found. Researchers posed various medical questions to five major chatbots, discovering that approximately half of the responses were deemed "problematic." This issue stems from chatbots potentially using biased or incomplete training data, leading them to generate responses that prioritize aligning with user beliefs over factual accuracy.

The study highlighted that all AI types were implicated, with Grok exhibiting the most issues at 58%, followed by ChatGPT at 52% and Meta AI at 50%. The performance varied across medical topics, with chatbots performing best on vaccines and cancer but worst on stem cells, athletic performance, and nutrition. Researchers noted that chatbots do not access real-time data and lack the ability to reason or weigh evidence, making their authoritative-sounding responses potentially flawed.

Given these findings, experts stress the critical need for diligent oversight when incorporating AI chatbots into medicine. They are not licensed to provide medical advice and may not possess up-to-date medical knowledge. The study, published in BMJ Open, concluded that as AI use grows, public education, professional training, and regulatory oversight are essential to ensure generative AI supports, rather than undermines, public health.