Clinical Scorecard: Shadow AI and the Constraints on Health Care Institutions’ Ability to Adapt

At a Glance

Key Highlights

• The Swedish qualitative study identified clinical decision support, administrative work, research, and technological exploration as four purposes of Shadow AI use.
• Physicians reported using personal devices and accounts, including tools such as ChatGPT.
• The article reports that Shadow AI can address deficiencies in institutional systems, lessen cognitive workload, and provide access to knowledge difficult to obtain through formal channels.
• Many tools in use lack authorization as medical devices under the European Union Medical Device Regulation.
• The article describes potential institutional legal risks involving protected health information privacy and deployment of tools whose clinical validity may not be established.

Guideline-Based Recommendations

Diagnosis

Management

Monitoring & Follow-up

Risks

• Shadow AI creates legal risks for health systems concerning protected health information privacy and deployment of unapproved technology whose clinical validity may not have been established.
• The article states that rapid deployment without transparency may cause harm and that one highly publicized failure or adverse event may render a technology unacceptable for years.

Patient & Prescribing Data

No patient population or prescribing regimen is specified; the central empirical study surveyed 357 physicians in Sweden.

The article discusses clinician use of AI tools, not a specific treatment. It reports that Shadow AI is used for clinical decision support, administrative work, research, and technological exploration.

Clinical Best Practices

• The article identifies institutional governance capacity as a constraint on clinicians’ adoption of AI; it does not specify a clinical best-practice protocol.
• The article discusses transparency and enduring community trust in relation to technology deployment; it does not prescribe a best-practice protocol.

Related Resources & Content

• Petersson et al. Shadow AI in Swedish health care: qualitative analysis of physicians’ free-text answers
• McDuff et al. Towards accurate differential diagnosis with large language models
• Rader et al. Meeting clinicians where they are through signal-guided AI innovation
• Baillette et al. Benefits and risks of shadow IT in health care: a narrative review of the literature
• Rader et al. Trends in perception of generative AI in healthcare by US adult demographics and political identity
• Frith. Letter to Layla Moran MP, Chair of the Health and Social Care Committee
• NHS England. NHS Federated Data Platform uptake and benefits
• Moran. Letter from the Chair to the Minister of State for Health Innovation and Safety on the NHS Federated Data Platform