To present a technical feasibility report on a user interface-based sandbox developed for testing AI tools in patient portal message management within a secure research environment.
Approach:
Setting and Governance: Development and experiments were conducted in a secure research environment with institutional review board (IRB) approval, ensuring no production EHR systems were affected.
Software Stack and Architecture: The system was implemented using Python and Streamlit, featuring a modular backend for data handling, privacy, and AI task execution, with core packages for natural language processing and entity recognition.
Key Findings:
The sandbox allows for safe testing and iteration of AI outputs without impacting live EHR systems.
It supports secure experiments with real messages while ensuring patient safety and system stability.
The architecture is designed to facilitate updates and reproducibility.
Interpretation:
The developed sandbox provides a controlled environment for evaluating AI tools in patient portal communications, addressing the risks associated with direct integration into live EHR systems.
Limitations:
The sandbox is separate from live EHR systems, which may limit the applicability of findings to real-world settings.
The reliance on simulated data may not fully capture the complexities of actual patient interactions.
Conclusion:
The user-friendly interface and sandbox environment are essential for the rigorous testing of AI applications in patient portal communications, ensuring safety and effectiveness before deployment.
