RCT of a WeChat-Integrated AI System for Orthopedic Postoperative Management
Overview
This randomized controlled trial evaluated a GPT-4-based AI agent integrated into WeChat for postoperative management in orthopedic patients. The AI system demonstrated high reliability and accuracy, reduced patient anxiety, improved functional outcomes, and enhanced satisfaction compared to standard doctor-led care.
Background
Postoperative management is essential for optimizing recovery and patient satisfaction after orthopedic surgery. Traditional follow-up methods face challenges such as patient noncompliance and limited accessibility, which may delay recovery and worsen outcomes. Advances in large language models like GPT-4 enable personalized, context-aware patient support, potentially overcoming these barriers. However, rigorous clinical trials assessing AI-driven postoperative interventions remain scarce.
Data Highlights
Characteristic
AI Group (n=140)
Doctor Group (n=121)
p-value
Age (years)
46.6 ± 18.5
48.0 ± 17.7
0.54
Height (cm)
167.1 ± 9.6
165.6 ± 8.9
0.38
Weight (kg)
71.7 ± 15.3
72.0 ± 13.9
0.29
Hip surgeries (%)
23.6%
29.8%
0.26
Knee surgeries (%)
76.4%
70.2%
0.26
Arthroscopy (%)
Not specified
Not specified
0.80
Baseline knowledge score
5.6 ± 2.9
5.9 ± 2.0
Not specified
AI system recall
92.8%
AI system precision
94.5%
AI system coverage
88.3%
Factual accuracy of AI responses
93.7%
Hallucination rate
6.3%
Key Findings
The AI agent demonstrated high reliability with recall of 92.8%, precision of 94.5%, and coverage of 88.3% against expert-validated references.
Factual accuracy of AI responses in real-world patient interactions was 93.7%, with a low hallucination rate of 6.3%.
300 patients were randomized equally to AI and doctor groups, with comparable baseline demographics and surgical characteristics.
Follow-up retention was higher in the AI group (140 analyzed) versus the doctor group (121 analyzed).
The AI intervention reduced postoperative anxiety and improved functional and mental health outcomes compared to standard care.
Patient satisfaction was higher in the AI-supported group, indicating enhanced engagement and perceived support.
Clinical Implications
Integrating a GPT-4-based AI system into a widely accessible platform like WeChat can provide scalable, reliable postoperative support for orthopedic patients. This approach may overcome traditional barriers such as limited access and patient anxiety, leading to improved functional recovery and satisfaction. Clinicians should consider AI-assisted follow-up as a complementary tool to enhance postoperative care delivery.
Conclusion
This study provides robust evidence that a WeChat-integrated GPT-4 AI agent can safely and effectively improve postoperative management in orthopedic patients, reducing anxiety and enhancing outcomes compared to standard doctor-led care. Such AI-driven interventions hold promise for scalable, personalized postoperative support.
References
GPT-4 and LLM capabilities in medicine
Ethical and regulatory challenges of AI in clinical practice
Patients with preoperative vitamin D deficiency had higher postoperative pain scores and opioid use after mastectomy, including more than triple the odds of moderate to severe pain within 24 hours of surgery.
