To propose training clinical AI on decisions made by clinicians identified as providing best practice, rather than on all routine clinical decisions or outcomes.
Approach:
Problem: Routine decisions and outcomes can reflect clinician practice patterns and local care systems, so models trained on them may reproduce average practice.
Proposed strategy: Define best-practice clinicians a priori for the clinical problem and stakeholder priorities, then train models on cases they managed. Criteria could include risk-adjusted outcomes, resource use, peer or patient assessments, knowledge, experience, or a combina…
Example: For ICU triage, best-practice clinicians might be those with lower admission rates while maintaining non-inferior outcomes among admitted and non-admitted patients.
Key Findings:
In a hypothetical ICU example, average admission rates of 12% would standardise recommendations at that level, even if best-practice clinicians admitted 8%, leaving a four-percentage-point over-admission bias.
The authors propose that training on best-practice decisions could reduce both noise and bias, but this has not been established by outcome evidence in the article.
Interpretation:
The authors argue that the practice used as an AI model’s standard matters: reproducing prevailing decisions may standardise care without improving its appropriateness.
Limitations:
Apparent superior performance may reflect differences in case mix; rigorous risk-adjusted comparisons and comprehensive outcome measures are needed.
Best practice may depend on local resources and context, requiring local training or calibration and ongoing validation for changes over time.
Selecting fewer clinicians may reduce sample size and representation of patient subgroups; broader data access and fairness and subgroup audits are proposed.
Models require prospective evaluation of safety, efficacy, workflow, clinician behaviour, and patient outcomes; predictive performance alone is insufficient.
Implementation also requires AI literacy, defined oversight, and safeguards against automation bias.
Conclusion:
Further research is needed to determine whether best-practice clinicians can be identified reliably and whether models trained on their decisions improve outcomes.
