Assess the modeling choices, interpretability, validation, and reporting of Lyu et al.’s machine-learning framework for predicting poststroke anxiety (PSA).
Approach:
Model selection: The study compared seven algorithms. Logistic regression was selected based on better held-out test performance, the lowest reported test-set Brier score among the models, and more consistent calibration, despite slightly lower discrimination than stacking.
Variable selection and explanation: Univariate screening and LASSO reduced 49 candidate variables to 18. SHAP was then applied to the fitted logistic regression model; its importance ranking, alongside redundancy, cost, and bedside feasibility, informed selection of ten nomogram variables.
Validation and reporting review: The correspondence examines the small test partition, the absence of external validation, and inconsistencies in the article’s figures, text, and tables.
Key Findings:
The logistic regression model had a reported test AUC of 0.981 and was favored for calibration and test-set Brier score as well as discrimination.
The ten-variable nomogram includes NIHSS score, Barthel index, LDL cholesterol, sleep quality, and income level.
SHAP was used to interpret the fitted model and inform a parsimonious instrument; it did not perform variable selection or establish causal effects.
The test partition contained approximately 36 patients: 15 with PSA and 21 without. The reported sensitivity of 0.867 corresponds to 13 of 15 cases, and specificity of 0.952 to 20 of 21 non-cases.
All evaluation partitions came from the same multicenter cohort and recruitment window; the reported evaluation is internal rather than external validation.
The article contains discrepancies concerning cohort counts and prevalence, cross-validation folds, the test-set Brier score, and the stacking model’s test AUC.
Interpretation:
The correspondence describes the nomogram as a clinically motivated candidate instrument, not a decision-support tool ready for bedside use. It states that external testing in data differing by geography, time, or case mix is needed to assess transportability.
Limitations:
The test partition was small, making performance estimates sensitive to individual classification errors.
No external validation was performed; all evaluation data came from the study cohort.
Inconsistencies across the article’s figure, text, and tables may mislead readers about the cohort and reported results.
Conclusion:
The correspondence supports the authors’ choice of a simpler, calibrated, interpretable model while calling for prospective external validation and correction of reporting discrepancies.
