Large Language Model Summarization of Physician-to-Physician Calls for Interhospital Transfer of Patients With ST-Elevation Myocardial Infarction: Observational Study - Summary - MDSpire
Advertisement
Large Language Model Summarization of Physician-to-Physician Calls for Interhospital Transfer of Patients With ST-Elevation Myocardial Infarction: Observational Study
To assess the feasibility of transcription and large language model (LLM) summarization of STEMI transfer calls without human intervention, using the Physician Documentation Quality Instrument (PDQI) for evaluation.
Approach:
Data Acquisition: Identified patients transferred to VUMC for STEMI from January 1 to June 30, 2024, and reviewed transfer call recordings for clinical information.
Transcription and Summarization: Transcribed calls using OpenAI Whisper model and analyzed transcripts with aiChat, a HIPAA-compliant LLM.
Quality Measurement: Used a modified Physician Documentation Quality Instrument (PDQI) to evaluate the quality of LLM-generated summaries.
Statistical and Qualitative Analysis: Calculated interrater reliability and conducted thematic analysis of rater comments.
Key Findings:
The quality of communication between referring and receiving institutions is a major barrier to effective transfer, as identified in the study.
LLM-generated summaries can provide a consistent format for clinical details during transfers, as demonstrated by the analysis.
The study utilized a modified PDQI to assess the quality of LLM summaries, indicating a structured approach to evaluation.
Interpretation:
The study presents findings on the use of LLMs in enhancing communication in STEMI transfers, with a need for further evaluation.
Limitations:
The study was limited to a single institution and specific time frame.
Raters were not blinded to the study hypothesis, which may introduce bias.
Conclusion:
The study suggests that LLMs may improve the quality of information transfer during STEMI patient transfers, indicating a need for further investigation.
by Jesse O Wrenn, Madelaine Behrens, Mary S Hershey, Marc Maldaver, John Mitchell, Trevor Thompson, Austin J Triana, Zain M Virk, Yasemin Akdas, Michael R Cauley, Michael J Ward, Ken Monahan
