Large language models provide unsafe answers to patient-posed medical questions - Takeaways - MDSpire

Large language models provide unsafe answers to patient-posed medical questions

  • By

  • Rachel L. Draelos

  • Samina Afreen

  • Barbara Blasko

  • Tiffany L. Brazile

  • Natasha Chase

  • Dimple Patel Desai

  • Jessica Evert

  • Heather L. Gardner

  • Lauren Herrmann

  • Aswathy Vaikom House

  • Stephanie Kass

  • Marianne Kavan

  • Kirshma Khemani

  • Amanda Koire

  • Lauren M. McDonald

  • Zahraa Rabeeah

  • Amy Shah

  • February 13, 2026

  • 0 min

Share

  • 1

    The study evaluates the safety of four large language model chatbots in providing medical advice to patients.

  • 2

    A total of 888 responses to 222 medical inquiries were analyzed, revealing significant differences in safety among the chatbots.

  • 3

    The rate of problematic responses ranged from 21.6% for Claude to 43.2% for Llama, indicating varying levels of safety.

  • 4

    Unsafe responses were found in 5% of Claude's answers, while GPT-4o and Llama had unsafe rates of 13%.

  • 5

    The findings suggest that many patients may receive unsafe medical advice from these chatbots, necessitating further safety improvements.

Original Source(s)

Large language models provide unsafe answers to patient-posed medical questions

  • by Rachel L. Draelos, Samina Afreen, Barbara Blasko, Tiffany L. Brazile, Natasha Chase, Dimple Patel Desai, Jessica Evert, Heather L. Gardner, Lauren Herrmann, Aswathy Vaikom House, Stephanie Kass, Marianne Kavan, Kirshma Khemani, Amanda Koire, Lauren M. McDonald, Zahraa Rabeeah, Amy Shah

  • February 13, 2026

Related Content

Toxicant Exposure Echoes for Generations

Study tracks epigenetic inheritance, organ pathology, and reproductive complications across maternal and paternal lineages

  • March 3, 2026

  • 3 min
Low-Dose PFAS Exposure Alters Cellular Metabolism 

Cell culture model reveals lipid remodeling and oxidative stress following prolonged PFAS exposure 

  • March 24, 2026

  • 2 min
Accelerating Structure-based Drug Design for Complex Membrane Proteins

Nuclera and leadXpro partnership will deliver AI-guided end-to-end workflow to access challenging membrane protein targets

  • March 9, 2026

  • 3 min