Clinical Report: Impact of Erroneous AI on Radiologists in Lung Cancer Chest X-rays

Overview

This pilot study evaluated how incorrect AI outputs affect radiologist performance in lung cancer detection on chest radiographs and whether human factors modifications can mitigate these effects. Findings indicate that false AI results can mislead radiologists, but strategies such as deleting AI feedback or visually outlining suspicious areas may reduce errors.

Background

Artificial intelligence (AI) use in radiology has expanded rapidly, with many systems approved for clinical use and increasing adoption by radiologists. AI algorithms for lung cancer detection on chest imaging demonstrate good but imperfect accuracy, with notable false positive and false negative rates. While AI can improve radiologist performance overall, erroneous AI feedback may adversely influence diagnostic decisions. Understanding and optimizing human factors in AI implementation is critical to minimizing these negative impacts and improving patient outcomes.

Data Highlights

Key Findings

• AI systems for lung cancer detection on chest X-rays had a ROC-AUC of 0.87 with 8 false positives and 4 false negatives among 90 cases.
• Radiologists’ diagnostic performance was negatively impacted by incorrect AI feedback, leading to potential misdiagnoses.
• Deleting AI feedback from patient files reduced the deleterious influence of false AI results on radiologist decisions.
• Providing a visual box outlining the AI-identified suspicious area improved radiologist accuracy compared to AI feedback without a box.
• The order of AI feedback presentation was controlled to prevent contamination of radiologist interpretation across conditions.

Clinical Implications

Clinicians should be aware that erroneous AI outputs can bias radiologist interpretations, potentially leading to missed or incorrect diagnoses. Implementing human factors strategies such as removing incorrect AI feedback from patient records or visually highlighting AI-flagged abnormalities may mitigate these risks. Careful integration of AI tools with attention to feedback presentation is essential to optimize diagnostic accuracy and patient care.

Conclusion

Erroneous AI results can adversely affect radiologist performance in lung cancer detection on chest radiographs, but optimizing human factors in AI implementation—such as deleting AI feedback or using visual cues—can reduce this impact. These findings underscore the importance of thoughtful AI integration to enhance rather than hinder clinical decision-making.

References

1. ACR Data Science Institute and related studies (2020-2022) -- AI in Radiology Approvals and Usage
2. Alberdi et al. (Mammography CAD Study) -- Impact of False Negative AI Feedback on Radiologist Decisions
3. Review of 503 AI Studies (2020) -- AI Accuracy in Lung Nodule Detection
4. Human Factors Psychology in Radiology -- Optimizing AI Implementation