Personal Health Large Language Models and the Negotiation of Medical Authority in Clinical Care: Opportunities, Risks, and Governance - Report - MDSpire

Personal Health Large Language Models and the Negotiation of Medical Authority in Clinical Care: Opportunities, Risks, and Governance

  • By

  • Wenyi Xie

  • Jialin Liu

  • Siru Liu

  • June 25, 2026

  • 0 min

Share

Clinical Report: Navigating Medical Authority in Clinical Practice

Background

The integration of personal health large language models (PH-LLMs) into healthcare represents a significant shift in how diagnostic reasoning and therapeutic recommendations are formulated. As these models synthesize user-generated health data into personalized narratives, they challenge traditional clinical authority and accountability.

Data Highlights

No numerical or trial data is presented in the source material.

Key Findings

  • PH-LLMs differ from traditional health chatbots by providing ongoing interpretations and recommendations based on user data.
  • Current PH-LLMs may be vulnerable to issues such as data shift and hallucination, impacting their reliability.
  • Functions of PH-LLMs exist on a risk continuum, with varying implications for patient safety and clinical decision-making.
  • The use of PH-LLMs may shift clinician-patient interactions from a dyadic to a triadic model, involving negotiation of authority among clinicians, patients, and algorithms.
  • There are significant risks associated with PH-LLMs, including epistemic conflict and fragmentation of clinical truth.

Clinical Implications

Clinicians may need to adapt their approaches to accommodate the influence of PH-LLMs in patient interactions. This includes being prepared to mediate between algorithmic outputs and clinical evidence to ensure safe and effective patient care.

Conclusion

The emergence of PH-LLMs necessitates a reevaluation of medical authority and governance in clinical practice. A framework that prioritizes safety, accountability, and trust is essential for their beneficial use.

Related Resources & Content

  1. Journal of Medical Internet Research (JMIR), 2026 -- Ethical Governance of Large Language Models in Health Care: Trust, Responsibility, and Equity in Routine Use
  2. Journal of Medical Internet Research (JMIR), 2026 -- Human-in-the-Loop as a Safety Guardrail: Clinical Accountability in the Large Language Model Era
  3. BMJ Mental Health, 2026 -- Leveraging simulation to provide a practical framework for estimating the novel scope of risk of large language models in healthcare
  4. Journal of Medical Internet Research (JMIR), 2026 -- Ethical Considerations in Personal Health Large Language Models
  5. FDA, 2025 -- Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles
  6. Joint Commission International, 2026 -- Responsible Use of AI in Healthcare Certification
  7. ACR, 2026 -- First Practice Parameter for Imaging Artificial Intelligence
  8. Considerations for Generative AI in Public Health | Artificial Intelligence | CDC
  9. FTC Launches Inquiry into AI Chatbots Acting as Companions
  10. Large language model diagnostic assistance for physicians in a lower-middle-income country: a randomized controlled trial
  11. Mitigating Automation Bias in Physician-LLM Diagnostic Reasoning Using Behavioral Nudges: A Randomized Controlled Trial
  12. Human–large language model collaboration in clinical medicine: a systematic review and meta-analysis
  13. Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles | FDA
  14. Joint Commission Releases First of Its Kind Exclusively Designed for Healthcare Organizations, Voluntary Responsible Use of AI in Healthcare Certification | Joint Commission International
  15. ACR Approves First Practice Parameter for Imaging Artificial Intelligence
  16. | Health Industry Cybersecurity Third-Party AI Ris

Original Source(s)

Related Content