Multimodal Integration of Endoscopic and Radiomic Data for Predicting Survival Outcomes in Colorectal Cancer

Overview

HydraMamba, a novel multimodal selective state space framework, integrates endoscopic and CT imaging data to enhance lesion segmentation, detection, and survival prediction in colorectal cancer. The model achieved state-of-the-art performance metrics in lesion analysis and demonstrated well-calibrated survival modeling, highlighting the benefit of combining complementary imaging modalities.

Background

Colorectal cancer remains a major cause of morbidity and mortality, with accurate lesion characterization and staging critical for effective management. Endoscopy provides high-resolution mucosal visualization, while CT imaging offers deeper anatomical context, but these modalities have traditionally been analyzed separately. Recent advances in deep learning enable improved lesion detection and prognostication from medical images. Multimodal learning approaches that fuse endoscopic and radiologic data have shown promise in enhancing diagnostic accuracy and survival prediction by leveraging complementary information.

Data Highlights

Key Findings

• HydraMamba achieved state-of-the-art lesion segmentation and detection performance on both endoscopic and CT datasets, with Dice scores of 0.856 and 0.812 respectively.
• The model delivered high F1 scores (0.918 for endoscopy and 0.888 for CT), indicating strong precision and recall in lesion analysis.
• Survival prediction using CT data showed excellent calibration and discrimination, with Harrell’s C index of 0.832 and Uno’s C@1y of 0.853.
• The integrated Brier score of 0.161 and calibration slope near 1.01 demonstrate the model’s reliable prognostic performance.
• By fusing endoscopic and CT information within a shared state space framework, HydraMamba effectively captures complementary anatomical and pathological features.
• This multimodal approach outperforms single modality models, supporting the clinical value of integrated imaging analysis in colorectal cancer.

Clinical Implications

HydraMamba’s integration of endoscopic and CT imaging data provides clinicians with a more accurate and comprehensive tool for lesion detection and survival prognostication in colorectal cancer. This approach may enhance preoperative assessment, guide personalized treatment planning, and improve patient risk stratification. Adoption of such multimodal AI frameworks could reduce diagnostic uncertainty and support decision-making in complex oncologic cases.

Conclusion

The HydraMamba framework demonstrates that multimodal fusion of endoscopic and radiomic data can significantly improve lesion analysis and survival prediction in colorectal cancer. This unified model offers a promising foundation for advancing precision oncology through integrated imaging biomarkers.

References

1. Zhang et al. 2021 -- Deep learning fusion of CT and endoscopy for gastric cancer staging
2. Qian et al. 2022 -- Multimodal prognostic modeling in head and neck cancer
3. Recent advances in AI for colorectal lesion detection and segmentation