Neural Cellular Automata Enable On-Capsule Diagnosis and Navigation in WCE
Overview
This study introduces neural cellular automata (NCA) models for wireless capsule endoscopy (WCE), achieving superior bleeding segmentation accuracy and convincing depth estimation while being 10 times smaller than state-of-the-art models. The models were successfully ported to a microcontroller small enough to fit inside a capsule endoscope, enabling on-capsule diagnosis and localization.
Background
Wireless capsule endoscopy is a noninvasive technique for imaging the entire gastrointestinal tract, including the small intestine, offering a pain-free alternative to traditional endoscopy. However, reviewing hours of video data is time-consuming, and localizing the capsule within the GI tract remains challenging. Existing localization methods require additional hardware or cumbersome sensor arrays. Visual odometry using depth images derived from monocular depth estimation models offers a promising image-guided approach but is computationally intensive and currently limited to PC-based retrospective analysis.
Data Highlights
Metric
NCA Model
Other Small-Scale Models
Bleeding Segmentation Accuracy (Dice)
29.1% higher
Baseline
Model Size
10x smaller
Larger state-of-the-art models
Key Findings
Neural cellular automata (NCA) models achieve 29.1% higher Dice accuracy in bleeding segmentation compared to other small-scale models.
NCA models produce convincing depth maps from RGB images by distilling knowledge from large foundation models using pseudo-ground truth data.
The NCA-based segmentation and depth estimation models are sufficiently lightweight to be ported to an ESP32-S3 microcontroller, suitable for integration inside a capsule endoscope.
This approach enables on-capsule diagnosis and localization without requiring additional sensing hardware or external sensor arrays.
The study pioneers the use of NCA in capsule endoscopy, demonstrating their robustness and efficiency for medical imaging tasks in constrained hardware environments.
Clinical Implications
The integration of NCA models directly on capsule endoscopes can significantly reduce physician review time by enabling real-time, on-device detection of bleeding and localization within the GI tract. This advancement may improve diagnostic precision and patient comfort by eliminating the need for additional sensors or cumbersome external equipment. Ultimately, it paves the way for more autonomous and efficient capsule endoscopy procedures.
Conclusion
Neural cellular automata provide a promising lightweight and robust solution for enhancing diagnostic precision and navigation in wireless capsule endoscopy. Their successful deployment on microcontroller hardware marks a critical step toward fully autonomous, on-capsule medical imaging and localization.
References
Ozyoruk et al. 2023 -- EndoSLAM: Depth Estimation and Visual Odometry for WCE
Vats et al. 2022 -- Contrastive Learning for Bleeding Detection in WCE
Jeong et al. 2023 -- Simulated Data and CycleGAN for WCE Depth Estimation
