Abstract
Polyps are abnormal tissue growths in the colon that may develop into colorectal cancer if left undetected. Accurate segmentation in medical imaging is essential for early diagnosis and treatment. Although deep learning has greatly improved polyp segmentation, its dependence on large annotated datasets and substantial computational resources hampers generalization across diverse clinical settings. To overcome these challenges, we propose PRISM, a momentum-based self-distillation method that improves segmentation performance without introducing additional inference cost. Instead of storing or reusing past predictions, PRISM constructs a temporally smoothed teacher model by applying an exponential moving average (EMA) to the model's weights throughout training. This momentum-based teacher provides stable and adaptive supervision signals that co-evolve with the student model. We evaluate PRISM on colonoscopy datasets collected from five distinct medical centres and validate its generalization on an unseen independent dataset. PRISM achieves a Dice score of 0.81 and an IoU of 0.75, outperforming baseline and conventional self-distillation methods. Ablation studies confirm the effectiveness of the EMA-based teacher model in improving segmentation accuracy. PRISM offers a computationally efficient and generalizable solution for polyp segmentation tasks. The code is available at: https://github.com/TugberkErol/PRISM.