Clinical Report: Multimodal Human-Computer Interaction in Interventional Radiology and Surgery

Overview

This review systematically analyzes multimodal human-computer interaction (MMI) techniques applied in surgical and interventional radiology settings over the past decade. It highlights trends in input modalities, medical tasks addressed, and challenges encountered, providing a comprehensive overview of current approaches and future directions.

Background

Human-computer interaction (HCI) in medicine is critical for accessing information and documenting findings during sterile interventions where conventional input devices are impractical. Multimodal interfaces combine several input and output modalities, offering natural and flexible interaction suited for complex clinical environments. Despite advances in MMI across various fields, a focused analysis on surgical and radiological applications has been lacking. This review addresses this gap by systematically evaluating literature from 2013 to 2023.

Data Highlights

The final sample included 31 publications (16 journal articles, 15 conference papers) published between 2013 and 2023. The review identified trends in input modalities used, distribution of publications per year, and the medical tasks targeted by MMI systems. The literature search and screening process followed PRISMA guidelines, ensuring rigorous selection based on predefined inclusion and exclusion criteria.

Key Findings

• Input modalities have evolved over time, with increasing diversity and combination of sensory channels such as touch, vision, and voice.
• MMI systems were developed primarily for surgical and interventional radiology tasks requiring sterile, hands-busy environments.
• Common input modalities include gesture recognition, voice commands, and eye tracking, often combined to provide flexible interaction options.
• Output modalities frequently utilize visual, auditory, and haptic feedback to support multimodal communication with the user.
• Challenges identified include maintaining sterility, managing cognitive load, and ensuring reliable recognition of input in complex clinical settings.

Clinical Implications

Multimodal interfaces can enhance intraoperative workflow by enabling touchless, flexible interaction with computer systems, reducing contamination risk and accommodating hands-busy clinicians. Understanding the strengths and limitations of various input and output modalities can guide the design of more effective clinical HCI systems. Addressing identified challenges will be critical to successful implementation and adoption in surgical and radiological practice.

Conclusion

This comprehensive review underscores the growing importance and potential of multimodal human-computer interaction in interventional radiology and surgery. Continued research and development tailored to clinical needs are essential to optimize these technologies for real-world application.

References

1. Moher et al. 2009 -- PRISMA Statement
2. Atkinson et al. -- Standards for Systematic Reviews