SAM+D: Parameter-Efficient Dimensional Lifting of SAM-Family Models via Depth-Routed LoRA and Depth Shifting

Existing methods for adapting 2D foundation models such asSAM to 3D volumes either process slices independently—ignoring inter-slice context—or require substantial architectural changes and retraining.In this paper, we present SAM+D, a parameter-efficient framework thatlifts SAM-family models by one spatial dimension—enabling 3D volu-metric segmentation from 2D SAM and, for the first time via parameter-efficient fine-tuning, end-to-end 4D (3D+T) spatiotemporal segmenta-tion from video-based SAM2—while keeping the vast majority of pre-trained parameters frozen. SAM+D introduces two lightweight, model-agnostic modules into frozen transformer blocks: (1) Depth-RoutedLoRA (DRLoRA) experts with learned routing for spatially adaptivelow-rank updates, and (2) Depth Shift Modules (DSM) for cross-slice feature exchange at zero additional parameter cost. Together, theyprovide volume-level context while tuning only ∼2.8% of parameters forSAM and ∼3.7% for SAM2. We evaluate SAM+D in two distinct set-tings, each lifting the base model by one spatial dimension: 3D segmenta-tion, where SAM (2D → 3D) is evaluated on four CT benchmarks (KiTS,Pancreas, LiTS, Colon), and 4D segmentation, where SAM2 (2D+T →3D+T) is evaluated on a cell tracking challenge (CTC) dataset (Fluo-N3DH-SIM+). In both settings SAM+D achieves competitive or superiorresults under the single-point prompt setting while using fewer trainableparameters than existing methods, demonstrating that SAM+D general-izes across SAM-family architectures, target dimensionalities (3D, 4D),and domains spanning medical imaging and bio-scene understanding.Code is publicly available at https://github.com/JerrySongCST/SAM-Plus-D.