Clinical Report: ChatGPT-4 as Decision Support in Neuro-Oncology Radiotherapy
Overview
This prospective study evaluated ChatGPT-4’s ability to support radiotherapy decisions in 101 neuro-oncology cases. The AI showed high concordance with expert tumor board recommendations in low-complexity cases but demonstrated decreased accuracy in intermediate and high-complexity scenarios.
Background
Radiotherapy planning for central nervous system tumors requires integrating evidence-based guidelines with patient-specific factors, often involving multidisciplinary consensus. Errors in treatment planning can significantly affect outcomes and toxicity. Large language models like ChatGPT-4 have shown potential in medical knowledge tasks but their reliability in specialized domains such as neuro-oncology radiotherapy remains uncertain. This study aimed to assess ChatGPT-4’s concordance with expert decisions and clinical guidelines across varying case complexities.
Data Highlights
Case Complexity
Number of Cases
ChatGPT-4 Concordance with MTB (%)
Concordance with Guidelines (%)
Low Complexity
61
92%
95%
Intermediate Complexity
26
65%
N/A
High Complexity
14
43%
N/A
Key Findings
ChatGPT-4 achieved 92% concordance with tumor board decisions in low-complexity neuro-oncology radiotherapy cases.
Concordance dropped to 65% and 43% in intermediate and high-complexity cases, respectively, indicating challenges in ambiguous or guideline-lacking scenarios.
ChatGPT-4’s recommendations aligned closely with established guidelines (e.g., ESMO, NCCN) in straightforward cases.
Repeated queries after 30 days showed high consistency in ChatGPT-4’s responses, suggesting stable performance over time.
Discrepancies often involved under- or over-treatment recommendations, highlighting risks of AI hallucination or lack of nuanced clinical judgment.
Performance varied by tumor histology, with best results in glioblastoma and meningioma cases and lower accuracy in rare tumor types.
Clinical Implications
ChatGPT-4 can serve as a reliable adjunct for radiotherapy decision support in neuro-oncology when cases have clear guideline-based recommendations. However, caution is warranted in complex or borderline cases where AI may not capture nuanced clinical factors, underscoring the need for expert oversight. Integration of such AI tools should complement, not replace, multidisciplinary clinical judgment.
Conclusion
ChatGPT-4 demonstrates promising concordance with expert neuro-oncology radiotherapy decisions in low-complexity cases but shows limitations in more complex scenarios. These findings support cautious, supervised use of LLMs as decision support aids in specialized oncology workflows.
References
OpenAI/ChatGPT-4/2024 -- Evaluating ChatGPT-4 in Neuro-Oncology Radiotherapy Decision Support
