To investigate the effectiveness of dual-view log-mel spectrogram features and adaptive fusion strategies for automatic dysarthria detection during awake brain surgery.
Approach:
Research Questions: The study addresses the following research questions: 1) Does combining complementary encodings of the same log-mel spectrogram improve dysarthria detection compared to unimodal representations? 2) Are adaptive fusion strategies more reliable than simple feature concatenation in heterogeneous and noisy speech data? 3) How stable and sensitive are different fusion mechanisms in small clinical cohorts? 4) How robust are the models when generalizing from controlled laboratory recordings to intraoperative speech?
Key Findings:
Combining dual-view log-mel spectrogram features improves dysarthria detection compared to unimodal representations.
Adaptive fusion strategies such as attention and gating outperform simple feature concatenation in heterogeneous and noisy speech data.
The stability and data sensitivity of different fusion mechanisms vary, particularly in small clinical cohorts.
Models demonstrate varying robustness when generalizing from controlled laboratory recordings to intraoperative speech.
Interpretation:
The findings suggest that multimodal spectrotemporal fusion can enhance automatic speech monitoring in awake brain surgery settings.
Limitations:
The study is limited by the small size of the intraoperative dataset and the variability in patient speech characteristics.
Generalization to other clinical settings may require further validation.
Conclusion:
The study highlights the potential of advanced deep learning techniques for improving dysarthria detection during awake surgeries, emphasizing the need for robust models that can adapt to clinical variability.
