Cross-platform evaluation of LLM-generated educational texts on cardiac myxoma: quality, readability, and actionability using network analysis and latent profile analysis - Summary - MDSpire
Advertisement
Cross-platform evaluation of LLM-generated educational texts on cardiac myxoma: quality, readability, and actionability using network analysis and latent profile analysis
To evaluate the performance and limitations of widely used LLMs in generating patient education texts for cardiac myxoma, focusing on specific challenges such as readability and actionability.
Approach:
Key Findings:
Significant heterogeneity in information quality, understandability, and readability across generated texts, highlighting the need for improved educational resources.
Clinical actionability of the texts was generally low, indicating a gap in practical guidance for patients.
Longer texts correlated positively with higher informational quality scores, suggesting a trade-off between length and clarity.
Reading difficulty indices were negatively correlated with actionability, emphasizing the challenge of creating accessible materials.
Three text phenotypes identified: moderate-quality/low-readability, high-quality/high-actionability, and low-quality/easy-to-read, which can inform future content development.
Interpretation:
The application of LLMs in patient education for cardiac myxoma is limited by reading burden and insufficient behavioral guidance, which may hinder patient understanding and engagement.
Limitations:
Only a small subset of outputs approached a favorable quality–actionability profile, raising concerns about the overall utility of LLM-generated texts.
The study did not directly elicit real-world patient information needs, which may limit the relevance of the generated materials.
Conclusion:
Current LLM-generated educational materials for cardiac myxoma require improvements in readability and actionability.