EoS-FM: Can an Ensemble of Specialist Models act as a Generalist Feature Extractor?

Recent advances in foundation models have shown greatpromise in domains such as natural language processing and computervision, and similar efforts are now emerging in the Earth Observationcommunity. These models aim to generalize across tasks with limitedsupervision, reducing the need for training separate models for eachtask. However, current strategies, which largely focus on scaling modelsize and dataset volume, require prohibitive computational and data re-sources, limiting accessibility to only a few large institutions. Moreover,this paradigm of ever-larger models stands in stark contrast with theprinciples of sustainable and environmentally responsible AI, as it leadsto immense carbon footprints and resource inefficiency. In this work,we present a novel and efficient alternative: EoS-FM1 , an Ensemble-of-Specialists framework for building Remote Sensing Foundation Models(RSFMs). Our method decomposes the pre-training process into train-ing lightweight, task-specific ConvNeXtV2 specialists that can be frozenand reused. This modular approach offers strong advantages in efficiency,interpretability, and extensibility. Moreover, it naturally supports feder-ated training, pruning, and continuous specialist integration, making itparticularly well-suited for collaborative and resource-constrained set-tings. Our framework sets a new direction for building scalable and effi-cient RSFMs.