LLM Explanations: Steps Matter in Radiology
Radiologists assigned to receive step-by-step explanations from a large language model achieved higher diagnostic accuracy in a randomized vignette study, while differential-diagnosis outputs may have increased inappropriate reliance on incorrect model suggestions.
By
Andrea Surnit
May 7, 2026
Clinical Scorecard: LLM Explanations: Steps Matter in Radiology
At a Glance
Category Detail
Condition Diagnostic accuracy in radiology Key Mechanisms Use of large language models (LLMs) for diagnostic support Target Population Radiologists Care Setting Radiology departments
Key Highlights
Chain-of-thought support improved diagnostic accuracy by 12 percentage points over control. GPT-4 achieved 80% accuracy with chain-of-thought prompting. Differential-diagnosis support did not significantly improve accuracy compared to no LLM assistance. Radiologists were more likely to override incorrect LLM recommendations with chain-of-thought support. Findings are based on a controlled vignette setting, not routine clinical practice. Guideline-Based Recommendations
Diagnosis
Consider using chain-of-thought prompting to enhance diagnostic accuracy. Management
Integrate LLMs with chain-of-thought explanations in radiology workflows. Monitoring & Follow-up
Evaluate the impact of LLM recommendations on diagnostic decisions. Risks
Be cautious of reliance on differential-diagnosis outputs, as they may lead to following incorrect suggestions. Patient & Prescribing Data
Not specified; study focused on radiologists' performance.
No direct patient outcomes evaluated; focus on diagnostic accuracy.
Clinical Best Practices
Utilize chain-of-thought explanations to improve diagnostic reasoning. Encourage critical evaluation of LLM outputs among radiologists. Related Resources & Content
