Objective:

To analyze the inclusion of randomized clinical trial data in FDA-cleared artificial intelligence medical devices and assess their benefit-risk reporting.

Approach:

• Study Design: Cross-sectional analysis of AI and machine learning-enabled medical devices cleared by the FDA from September 1995 to July 2023.
• Data Sources: Utilized FDA decision summaries, device approvals databases, and Manufacturer and User Facility Device Experience and Medical Device Recalls databases.

Key Findings:

• Fewer than 2% of devices included randomized clinical trial data.
• 691 FDA-cleared AI/machine learning devices identified, primarily class II and cleared via the 510(k) pathway.
• Most devices were in radiology (76.9%), followed by cardiovascular medicine (10.1%) and neurology (2.9%).
• Study design was not described for 46.7% of devices, and sample size was missing for 53.3%.
• Only 6 devices (1.6%) cited randomized clinical trial data; 53 (7.7%) included prospective studies.
• Clinical performance outcomes were rarely reported, with sensitivity provided for 24% (166 devices) and specificity for 22% (152 devices).
• Demographic characteristics were disclosed in 4.5% (31 device summaries).
• Postmarket surveillance revealed 489 adverse events associated with 36 devices, including 1 death.

Interpretation:

Limitations:

• Limited reporting on study design and sample sizes for a significant proportion of devices.
• Low percentage of devices reporting clinical performance outcomes.

Conclusion:

Sources:

• JAMA Health Forum