Clinical Scorecard: Evaluating the Real-World Application of Deep Learning Technologies in Healthcare: A Systematic Review Based on Implementation Science Principles

At a Glance

Key Highlights

• 20 prospective studies included across radiology, otolaryngology, dermatology, and ophthalmology demonstrating DL effectiveness and feasibility in clinical workflows
• Most studies evaluated adoption and appropriateness; very few assessed implementation costs and none assessed sustainability
• Stakeholder acceptability was evaluated in less than half of the studies, highlighting a gap in understanding user perspectives

Guideline-Based Recommendations

Diagnosis

• Utilize DL systems validated in prospective real-world studies to support clinical decision-making in relevant specialties

Management

• Integrate DL tools into existing clinical workflows to enhance diagnostic accuracy and efficiency
• Adopt hybrid effectiveness-implementation study designs to guide deployment and assess both clinical and implementation outcomes

Monitoring & Follow-up

• Regularly evaluate adoption rates and appropriateness of DL tools in clinical practice
• Assess stakeholder acceptability to inform ongoing implementation strategies

Risks

• Limited data on implementation costs and sustainability may affect long-term deployment
• Potential gaps in stakeholder acceptance could hinder effective adoption

Patient & Prescribing Data

Patients in radiology, otolaryngology, dermatology, and ophthalmology settings undergoing diagnostic evaluation

DL systems have demonstrated clinical effectiveness and feasibility but require further real-world evaluation for cost, sustainability, and acceptability

Clinical Best Practices

• Employ implementation science frameworks to systematically evaluate DL tool deployment
• Focus on measuring adoption, appropriateness, and stakeholder acceptability during implementation
• Prioritize hybrid study designs combining clinical effectiveness and implementation outcomes for future research
• Address gaps in cost evaluation and sustainability planning to support long-term DL integration

References

• Abramoff et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy
• Ipp et al. Pivotal Evaluation of an Artificial Intelligence System for Autonomous Detection of Referrable and Vision-Threatening Diabetic Retinopathy
• Tham et al. Detecting visually significant cataract using retinal photograph-based deep learning
• Tseng et al. Considerations for Artificial Intelligence Real-World Implementation in Ophthalmology
• Young et al. Patient and general public attitudes towards clinical artificial intelligence: a mixed methods systematic review