Clinical Report: Evaluation and Development of a Safety-Oriented Chatbot

Overview

This feasibility study presents the development of a safety-oriented chatbot, 'Expert', designed to enhance women's sexual well-being through personalized health education. The study emphasizes the importance of integrating safety and risk governance into chatbot design to mitigate potential misinformation and ensure appropriate user support.

Background

The integration of large language models (LLMs) in health education has transformed how information is delivered, particularly in sensitive areas such as sexual health. However, concerns regarding the accuracy of chatbot responses and the potential for misinformation highlight the need for structured safety measures. This study addresses these challenges by developing a chatbot that prioritizes clinical safety and risk governance in its design and operation.

Data Highlights

No numerical data or trial results were provided in the article.

Key Findings

• The 'Expert' chatbot was developed with a focus on safety, integrating clinical and product rules to guide responses.
• A cross-functional team of 28 contributors was involved in the development process, ensuring diverse expertise in clinical, technical, and governance areas.
• The chatbot features a medically verified knowledge base and a recap to reinforce critical safety rules.
• A secondary LLM, termed 'Critic', was implemented to assist in semi-automated evaluation of chatbot responses for potential issues.
• The study emphasizes the need for proactive safety measures in the design of health chatbots, particularly in sensitive domains like sexual health.

Clinical Implications

Healthcare professionals should consider the integration of safety-oriented chatbots in sexual health education to provide accurate and supportive information. The use of structured frameworks for chatbot development can enhance user trust and safety, particularly in sensitive health contexts.

Conclusion

The development of the 'Expert' chatbot represents a significant step towards ensuring safety and accuracy in digital health education for women's sexual well-being. Future implementations should continue to prioritize safety and risk governance in chatbot design.

References

1. Frontiers in Digital Health, 2026 -- Chatbots as Frontline Educators in Sexual Reproductive Health Rights: Evidence, Limitations, and Ethical Considerations
2. Frontiers in Digital Health, 2026 -- Co-creating a program theory and evaluability assessment for an Irish single-session, synchronous chat-based youth mental health intervention: implications for outcome evaluation
3. Comparing Accuracy, Completeness, and Preferences of Physician and AI-Generated Communications in Urology: Insights from Patients and Doctors
4. Final Recommendation Statement: Screening for Intimate Partner Violence and Abuse of Older or Vulnerable Adults | United States Preventive Services Taskforce
5. CDC Clinical Guidelines on the Use of Doxycycline Postexposure Prophylaxis for Bacterial Sexually Transmitted Infection Prevention, United States, 2024 | MMWR
6. BMC Psychiatry (Springer) — Exploring the Role of Agency in Psychotherapy: Experiences of Individuals with Mental Health Challenges in Interactions with AI Chatbots and Human Therapists
7. Final Recommendation Statement: Screening for Intimate Partner Violence and Abuse of Older or Vulnerable Adults | United States Preventive Services Taskforce
8. CDC Clinical Guidelines on the Use of Doxycycline Postexposure Prophylaxis for Bacterial Sexually Transmitted Infection Prevention, United States, 2024 | MMWR
9. Evaluation and management of hypoactive sexual desire disorder in women. Recommendations from the 5th International Consultation on Sexual Medicine (ICSM 2024) | Sexual Medicine Reviews | Oxford Academic