LESV:Language Embedded Sparse Voxel Fusion for Open-Vocabulary 3D Scene Understanding
Abstract
Recent advancements in open-vocabulary 3D scene under-standing heavily rely on 3D Gaussian Splatting (3DGS) to register vision-language features into 3D space. However, we identify two critical limi-tations in these approaches: the spatial ambiguity arising from unstruc-tured, overlapping Gaussians which necessitates probabilistic feature reg-istration, and the multi-level semantic ambiguity caused by pooling fea-tures over object-level masks, which dilutes fine-grained details. To ad-dress these challenges, we present a novel framework that leverages SparseVoxel Rasterization (SVRaster) as a structured, disjoint geometry repre-sentation. By regularizing SVRaster with monocular depth and normalpriors, we establish a stable geometric foundation. This enables a de-terministic, confidence-aware feature registration process and suppressesthe semantic bleeding artifact common in 3DGS. Furthermore, we resolvemulti-level ambiguity by exploiting the emerging dense alignment proper-ties of the AM-RADIO foundation model, avoiding the computationaloverhead of hierarchical training methods. Our approach achieves state-of-the-art performance on Open Vocabulary Point Cloud Understanding,and highly competitive results on 3D Object Retrieval benchmarks.