Diagnostic performance of an open-source artificial intelligence model for detecting chest infections on after-hours emergency department chest radiographs in Singapore - Summary - MDSpire
Assess CheXNeXt’s diagnostic performance for identifying infection-related findings on after-hours emergency department chest radiographs.
Approach:
Design and setting: Retrospective assessment of 1,006 anonymised after-hours emergency department chest radiographs.
Evaluation: CheXNeXt generated probabilities for consolidation, pneumonia and pleural effusion, which were compared with radiologists’ reports as the reference standard. The study calculated sensitivity, specificity, positive and negative predictive values, and area under the receiver operating characteristic curve (AUC), using published thresholds and thresholds optimised by the Youden index.
Key Findings:
At published thresholds, sensitivity was 0.234 for consolidation, 0.176 for pneumonia and 0.622 for pleural effusion; specificity was 0.990, 0.995 and 0.949, respectively.
AUC was 0.900 for consolidation, 0.856 for pneumonia and 0.916 for pleural effusion.
After Youden index threshold optimisation, sensitivity was 0.894 for consolidation, 0.681 for pneumonia and 0.852 for pleural effusion, while specificity was 0.765, 0.881 and 0.848, respectively.
Interpretation:
At default thresholds, CheXNeXt showed high specificity and limited sensitivity. The authors report that threshold optimisation enhanced performance and state that calibration is needed before clinical deployment. They describe potential for supporting triage and reducing emergency department chest radiograph workloads.
Limitations:
Radiologists’ reports were the reference standard, and reporting involved a single radiologist.
Radiographic findings were used as a surrogate for infection.
The study was conducted at one hospital; the authors call for external validation and real-world integration studies to assess generalisability and clinical impact.
Conclusion:
The authors conclude that CheXNeXt showed favourable performance in this setting, while further validation and evaluation in real-world workflows are needed.
