Evaluation of Four Advanced Language Models in Responding to Family-Centered Inquiries on Morbidity and Outcomes for Patients Undergoing Extracorporeal Membrane Oxygenation

Overview

This study evaluates the performance of four advanced language models (LLMs) in answering family-centered questions regarding morbidity and outcomes for patients undergoing ECMO.

Background

Extracorporeal Membrane Oxygenation (ECMO) is a critical life support system used in severe respiratory and cardiac failure, with increasing utilization during the COVID-19 pandemic. Families often seek accurate information about ECMO from various sources, including advanced language models.

Data Highlights

Key Findings

• All LLMs acknowledged the high-risk nature of ECMO and differentiated between VV-ECMO and VA-ECMO.
• LLMs reported shorter VV-ECMO support durations compared to ELSO-registry data.
• Discrepancies in complication rates were noted between LLM responses and ELSO-registry data.
• Gemini achieved the highest scores in both content and expert-perceived communicative quality.
• All LLMs provided disclaimers advising consultation with healthcare providers for individualized information.

Clinical Implications

The study highlights the importance of critically evaluating information provided by LLMs regarding ECMO outcomes. Healthcare providers should guide families in interpreting this information and emphasize the necessity of consulting medical professionals for personalized advice.

Conclusion

The evaluation of LLMs reveals significant variations in the quality of information provided, underscoring the need for careful consideration when utilizing these tools for family-centered communication in ECMO contexts.

Related Resources & Content

1. Fernando et al., Frontiers in Medicine, 2026 -- Comorbidities in extracorporeal membrane oxygenation: a comorbidome-based analysis
2. Intensive Care Medicine, 2022 -- Mortality Prediction Models for Patients Undergoing ECMO: A Systematic Review of Their Characteristics and Performance
3. Intensive Care Medicine, 2023 -- ECMO Survival Prediction: Implementing Deep Learning Models in Venoarterial Extracorporeal Membrane Oxygenation
4. SCCM, 2024 -- Guidelines on Family-Centered Care for Adult ICUs
5. Pediatric Cardiology — Factors Influencing Outcomes in Pediatric Extracorporeal Cardiopulmonary Resuscitation
6. Guidelines on Family-Centered Care for Adult ICUs: 2024 | SCCM
7. ECMO for severe ARDS: systematic review and individual patient data meta-analysis - PMC
8. Association between hyperoxia and mortality in patients undergoing extracorporeal membrane oxygenation: A systematic review and meta-analysis - ScienceDirect