AI chatbots have quietly become a go-to source for medical advice the extent that some are dangerously skipping real doctors. But when AI‘s health advice goes wrong, doctors may never see the exchange behind it.
Researchers from Binghamton University, Stanford, Texas A&M, and Indiana University published a paper in Nature Health examining what happens after a chatbot gives you bad health information. Their biggest concern is not just that chatbots can be wrong. It is that those mistakes often leave no trace for doctors, researchers, or regulators to catch.
How can harmful AI advice go unseen?
Nadeem Sarwar / Digital Trends
The researchers point to a real example to make their case. A 60-year-old man asked ChatGPT how to cut chloride from his diet and was told to swap table salt for sodium bromide. He followed that advice and ended up hospitalized with bromide toxicity, dealing with hallucinations and paranoia.
Because the chatbot conversation stayed locked on the AI platform, his doctors couldn’t see what he had actually been told. That made it nearly impossible to piece together what went wrong. Lead author Kaicheng Yang says this gap is exactly the problem, since no one is actively monitoring these interactions for errors.
Medical advice can be documented in charts, while harmful drug effects can enter reporting systems. Chatbot conversations, however, may remain accessible only to the user and AI companies. That leaves clinicians, regulators, and independent investigators without the evidence needed to identify patterns.
You can receive bad health advice without even asking questions
The paper also flags a less obvious risk. You do not need to ask a chatbot anything to be exposed to bad health information, since AI summaries and social feeds can surface it while you scroll.
What solutions do the researchers propose?
To fix this, the researchers want AI companies to give users access to their own chat history and build systems to flag harmful advice. They also urge policymakers to consider extending physician malpractice liability to AI chatbot companies providing medical guidance.
RELATED COVERAGE: