Clinical Scorecard: Updated AI/ML-Based Medical Devices Cleared via the FDA 510(k) Pathway
At a Glance
Category
Detail
Condition
Postclearance updates to AI/ML-based medical devices cleared through the FDA 510(k) pathway
Key Mechanisms
Updates may alter device use through new features or changes to algorithm inputs, or improve algorithm performance through new data, neural networks, or architecture changes.
Target Population
163 AI/ML-based medical devices initially cleared via the 510(k) pathway from May 12, 2015, through December 29, 2021, included in this cross-sectional analysis.
Care Setting
FDA regulatory review and postclearance oversight; 89.0% of devices in the study were designed to analyze medical imaging data.
Key Highlights
Among 163 devices, 56 (34.4%) had at least one AI/ML algorithm update recorded in FDA 510(k) database entries.
The median interval to the first recorded update was 1.8 years (IQR, 0.9-2.5).
Among updated devices, 42 of 56 (75.0%) had at least one update that altered use relative to the preceding version.
Among 106 update-related entries, participant information was missing from 41 (38.7%) and sample size was missing from 38 (35.8%).
The authors recommend enhancing and standardizing reporting requirements and establishing and enforcing postdeployment monitoring.
Guideline-Based Recommendations
Diagnosis
Not applicable: this cross-sectional study examined FDA database records of AI/ML device updates, not diagnosis of a disease.
Management
Author recommendation: enhance and standardize reporting requirements for AI/ML device modifications and their performance.
Author recommendation: establish and enforce postdeployment monitoring to support ongoing device safety and effectiveness.
Monitoring & Follow-up
The study assessed FDA documentation of performance evaluations, including participant information, sample size, metrics, results, and organizational or clinical benefits.
The authors identify postdeployment monitoring as a needed component of ongoing oversight.
Risks
The authors state that heterogeneous or incomplete reporting may limit physicians’ and patients’ ability to evaluate and select AI/ML-based devices.
The analysis may not capture updates made without an associated FDA modification request, including changes permitted under FDA Predetermined Change Control Plan guidance.
Patient & Prescribing Data
No patient treatment cohort was studied. The analysis covered 163 AI/ML-based devices cleared from May 12, 2015, through December 29, 2021.
The study did not evaluate treatment. It found that performance-assessment and benefit information for device updates was often absent from FDA database entries.
