To introduce a framework for assessing the operational safety of clinical AI systems beyond conventional accuracy metrics.
Key Findings:
The SA-ROC framework reveals that a model with a higher AUC may not be operationally safer.
In a case study of two FDA-cleared cancer screening algorithms, the one with superior AUC was less safe for high-confidence screening.
Interpretation:
The SA-ROC framework allows for active governance in clinical AI, translating policy into workflows that enhance operational safety and complement regulatory evaluations.
Limitations:
The framework's applicability may vary across different clinical contexts.
Further validation is needed to generalize findings across diverse AI systems.
Conclusion:
The SA-ROC framework provides a structured approach to assess and ensure the safety of AI systems in clinical settings, promoting responsible AI adoption.
Court-documented maltreatment was associated with small differences in GrimAge and DunedinPACE in sixth decade of life, but neither difference was statistically significant.
