To systematically review multimodal digital pathology techniques applied in colorectal cancer (CRC) and assess their performance compared to specific foundation models, such as traditional histopathology.
Key Findings:
Majority of studies integrated different modalities to enhance diagnostic accuracy and survival prediction.
Specific fusion techniques, such as early, intermediate, and late fusion, were employed to extract novel features.
Most studies lacked external validation.
Multimodal approaches showed superior performance compared to unimodal models.
Interpretation:
Multimodal models combining WSIs with clinical and genomic data significantly improve CRC diagnosis and prognosis, but challenges in data integration, such as managing heterogeneity and ensuring model interpretability, remain.
Limitations:
Limited number of studies included in the review, which may affect the generalizability of the findings.
Lack of external validation in most studies, raising concerns about the robustness of the results.
Challenges in managing data heterogeneity and modality weighting that could impact model performance.
Conclusion:
The review highlights the potential of multimodal approaches in CRC care, emphasizing the need for further research to address existing challenges and improve patient outcomes.
by Jitendra Jonnagaddala, Miljana Shulajkovska, Anton Gradišek, Toni Rose Jue, Qifeng Zhou, Yuzhi Guo, Jamil Mahmoud El Chayeb, Ruijiang Li, Jana Lipkova, Jakob Nikolas Kather, Junzhou Huang
