Clinical Scorecard: Evaluating the Quality of Responses from Large Language Models on Periprosthetic Fractures: A Comparison of ChatGPT, Gemini, and Claude
At a Glance
Category
Detail
Condition
Periprosthetic Fractures
Key Mechanisms
Assessment and treatment informed by implant stability, bone stock, and fracture configuration.
Target Population
Patients undergoing total hip and/or knee arthroplasty.
Care Setting
Tertiary university hospital
Key Highlights
LLMs can produce fluent text but may convey inaccurate information.
Periprosthetic femoral fractures are a major complication after total hip arthroplasty.
Standardized evaluation frameworks like Vancouver classification and UCS are used for treatment selection.
The study compared responses from ChatGPT, Gemini, and Claude to standardized patient questions.
Expert panel assessed the quality of responses based on accuracy, safety, and readability.
Guideline-Based Recommendations
Diagnosis
Use standardized questions to assess common patient concerns regarding periprosthetic fractures.
Management
Consider implant stability, bone stock, and fracture configuration in treatment decisions.
Monitoring & Follow-up
Evaluate responses for accuracy and safety when using LLMs for patient education.
Risks
Be aware of the potential for LLMs to provide misleading information.
Patient & Prescribing Data
Patients with periprosthetic fractures post-arthroplasty.
Responses from LLMs should be critically evaluated for accuracy.
Clinical Best Practices
Utilize established classification systems for periprosthetic fractures.
Engage in thorough assessment of patient inquiries before providing information.
