Objective:

To present a framework for implementing AI chatbots that can address patient distress and suicide risk in high-risk healthcare settings, particularly for individuals with substance use disorders.

Approach:

• Development of AI Chatbot 'Suzy': The AI chatbot 'Suzy' was developed to provide support for substance use disorder recovery and general wellness, with a focus on safety monitoring and suicide risk assessment.
• Safety Monitoring Techniques: Implemented a safety disclaimer and a 'Safety Router' to triage user messages based on risk levels, ensuring appropriate responses to potential psychiatric distress.
• Iterative Design Refinement: Incorporated feedback from clinicians and experts to refine the chatbot's ability to identify and respond to messages indicating varying levels of risk.

Key Findings:

• AI chatbots can provide on-demand behavioral health support but carry risks such as inconsistent responses and potential harm.
• The implementation of safety guardrails is crucial for monitoring suicide risk in AI chatbots.
• The design of 'Suzy' included a structured approach to assess and respond to user messages based on their risk levels.

Interpretation:

The study emphasizes the need for rigorous safety protocols in AI chatbots to effectively support individuals at risk of suicide, particularly in addiction medicine settings.

Limitations:

• The pilot design requires empirical validation beyond initial studies.
• Limited information exists on the optimal methods for monitoring suicide risk in AI chatbots.

Conclusion:

The framework presented aims to enhance the safety and effectiveness of AI chatbots in addressing patient distress and suicidal thoughts in high-risk environments.

Sources:

• OpenAI
• Dimagi, Inc.