Understanding Agreement Between AI and Clinicians in American Society of Anesthesiologists (ASA) Classification

1

Lin et al reported high agreement between clinician-assigned ASA physical status and classifications generated by a GPT-4o-powered preanesthetic system.
2

Because ASA physical status is ordinal, the report should specify whether Cohen κ was weighted to account for the severity of disagreements.
3

Among 600 patients, 498 (83%) were classified as ASA II, 57 as ASA III, and 6 as ASA IV; patient-level cross-tabulation was not provided.
4

Agreement increased from κ = 0.518 without Taiwan National Health Insurance data to κ = 0.883 with the data, a difference of 0.365.
5

The study provided no uncertainty estimate for the κ difference, although both evaluations involved the same 600 patients.

1