The potential of LLMs in generating questions and answers with EHRs
-
By
-
Yunqi Zhu
-
Wen Tang
-
Huayu Yang
-
Jinghao Niu
-
Liyang Dou
-
Yifan Gu
-
Yuanyuan Wu
-
Wensheng Zhang
-
Ying Sun
-
Xuebing Yang
-
July 15, 2026
Clinical Scorecard: Exploring the Capabilities of Large Language Models in Formulating Questions and Answers from Electronic Health Records
At a Glance
| Category | Detail |
|---|
| Condition | Large Language Models in Medical Education |
| Key Mechanisms | Utilization of LLMs to generate questions and answers from EHRs |
| Target Population | Medical students and interns |
| Care Setting | Clinical education and training |
Key Highlights
- LLMs can generate medical exam questions and answers from EHRs.
- ERNIE 4 outperformed other LLMs in question generation.
- Human experts scored higher in sufficiency of key information.
- LLMs showed higher information correctness compared to human experts.
- Doubao excelled in coherence and factual consistency for answer generation.
Guideline-Based Recommendations
Diagnosis
Management
Monitoring & Follow-up
Risks
Patient & Prescribing Data
Elderly patients with chronic diseases
LLMs may serve as auxiliary tools in medical education.
Clinical Best Practices
- Ensure coherence and professionalism in AI-generated content.
- Address biases and interpretability in LLM outputs.
- Utilize multimodal information to enhance LLM effectiveness.
Related Resources & Content
