FSD-Net: Foundation-Guided Spatiotemporal Distillation for Video Polyp Segmentation

Video polyp segmentation plays a pivotal role in early col-orectal cancer diagnosis, yet its clinical deployment is heavily bottle-necked by the prohibitive cost of dense pixel-level annotations. To addressthis challenge, we propose a foundation-guided spatiotemporal distilla-tion framework named FSD-Net for sparsely annotated video polyp seg-mentation. Our approach introduces a Semantic Flow Distillation (SFD)module that leverages a frozen foundation model as a teacher to extractrobust semantic features and generate continuous supervision signals fora lightweight student network. To tackle the frequent target disappear-ance caused by complex intestinal folds, we design an Occlusion-AwareCycle Consistency (OACC) mechanism that dynamically blocks erro-neous gradient propagation based on visibility estimation. Furthermore,we introduce a Prototype Re-identification Module (PRM) to maintainlong-term memory and successfully recapture target identities upon reap-pearance. Extensive experiments on three standardized datasets, includ-ing SUN-SEG, CVC-612, and CVC-300, demonstrate that our frame-work significantly outperforms existing weakly supervised methods andeven surpasses state-of-the-art fully supervised models. Notably, on thehighly challenging SUN-SEG-Hard dataset, our method achieves a mDice87.04%, mIoU of 79.20%, and mHD of 20.64 mm, proving its exceptionalrobustness and label efficiency in complex endoscopic environments.