Performance of four large language models in addressing family-centered questions on morbidity and outcomes in patients supported by extracorporeal membrane oxygenation - Summary - MDSpire
Advertisement
Performance of four large language models in addressing family-centered questions on morbidity and outcomes in patients supported by extracorporeal membrane oxygenation
To investigate the performance of four commonly used large language models (LLMs) in answering family-centered questions about morbidity and outcomes for adult patients undergoing ECMO.
Approach:
Key Findings:
All LLMs acknowledged the high-risk nature of ECMO and differentiated between VV-ECMO and VA-ECMO.
Responses generally aligned with ELSO-registry data, but LLMs reported shorter VV-ECMO support durations, specifically indicating a mean duration of X days compared to Y days in ELSO data.
Discrepancies in complication rates were noted between LLMs and ELSO-registry data.
Gemini achieved the highest average scores in both content and expert-perceived communicative quality.
A significant number of responses were rated unsatisfactory by ECMO experts.
Interpretation:
Limitations:
Questions were queried sequentially in a single chat session, which may inflate internal coherence.
Results reflect a snapshot of LLM capabilities in May 2025 and may not represent future performance.
Protection against spread appeared strongest within 6 months of vaccination, while exposed vaccinated contacts showed no measurable reduction in infection risk.
