Patient-facing diabetic foot information from large language models: a domain- and source-balanced prompt framework for public-interface benchmarking - Scorecard - MDSpire
Advertisement
Evaluating Patient-Centric Diabetic Foot Information Generated by Large Language Models: A Framework for Balanced Prompt Design and Public Interface Assessment
Clinical Scorecard: Evaluating Patient-Centric Diabetic Foot Information Generated by Large Language Models: A Framework for Balanced Prompt Design and Public Interface Assessment
At a Glance
Category
Detail
Condition
Diabetes-related foot disease
Key Mechanisms
Recognition of neuropathic risk, ulceration, infection, ischemia, offloading needs, and recurrence risk.
Target Population
Individuals with diabetes
Care Setting
Publicly accessible large language models for patient education
Key Highlights
Developed a 24-item benchmark prompt set for evaluating LLM outputs.
Significant differences in response quality observed across five LLMs.
Grok 4.3 achieved the highest scores for quality metrics.
No responses met all predefined readability targets.
Responses showed limited visible transparency and inadequate readability.
Guideline-Based Recommendations
Diagnosis
Timely recognition of neuropathic risk and ulceration.
Management
Emphasize prevention, early recognition, and multidisciplinary management.
Monitoring & Follow-up
Structured follow-up and escalation of care for suspected complications.
Risks
Potential for misinterpretation of LLM-generated information without clinician oversight.
Patient & Prescribing Data
Patients with diabetes at risk for foot disease.
Patient-facing information must be practical, understandable, and actionable.
Clinical Best Practices
Utilize a domain- and source-balanced prompt framework for patient education.
Ensure clinician oversight when using LLM-generated information for decision-making.
