AI Chatbots Are Even Worse at Giving Medical Advice Than We Thought
It’s tempting to think that an LLM chatbot can answer any question you pose it, including those about your health. After all, chatbots have been trained on plenty of medical information, and can regurgitate it if given the right prompts. But that doesn’t mean they will give you accurate medical advice, and a new study shows how easily AI’s supposed expertise breaks down. In short, they are even worse at it than I thought.In the study, researchers first quizzed several chatbots about medical information. In these carefully conducted tests, ChatGPT-4o, Llama 3, and Command R+ correctly diagnosed medical scenarios an impressive 94% of the time—though they were able to recommend the right treatment a much less impressive 56% of the time. But that wasn’t a real-world test for the chatbots medic...










