FedNASP: Federated Vision-Language Navigation with Adaptive Step-wise Personalization

Federated learning (FL) protects sensitive (Vision-LanguageNavigation) VLN data without centralizing trajectories or instructions,but severe non-IID environments make personalized FL (pFL) necessary.Moreover, VLN poses several coupled challenges for personalized feder-ated learning, including environment heterogeneity, multimodal language-vision fusion, and long-horizon navigation with time-varying decisioncontexts. To address these challenges, we propose FedNASP, a step-wisepersonalized federated learning framework for VLN. The key idea is todynamically calibrate personalization strength along a navigation trajec-tory. Specifically, we introduce a lightweight Step-wise Personalized Mod-ulator (SPM) that predicts personalization strength at each navigationstep. We further design a structure-aware adapter-based personalizedprefix injection mechanism that enables client-specific grounding whilekeeping the backbone shared across clients. Experiments on three rep-resentative datasets show that FedNASP consistently outperforms state-of-the-art federated VLN methods under substantial cross-client hetero-geneity. Compared with the non-centralized baselines, FedNASP improvesRemote Grounding Success on REVERIE by 13.0% and Success Rateon CVDN by 22.6%. Extensive ablation studies and visualizations fur-ther validate the effectiveness of adaptive step-wise personalization forfederated VLN. Code is available at: https://github.com/IntelliSys-Lab/FedNASP.git