LangLoc: “Tell Me What You See”

We tackle fine-grained indoor localization from natural lan-guage: given a free-form description of one’s surroundings, estimate theobserver’s 2D position and heading within a known 3D environment.Language queries are lightweight, privacy-preserving, and need no cam-era – yet prior work stops at coarse scene retrieval and cannot resolve anintra-scene pose. We close this gap with LangLoc, a three-stage pipelinethat (i) retrieves the correct scene via a dual-branch GATv2 encoderwith CLIP semantic features, surpassing the previous best by 8 per-centage points in Top-1 recall; (ii) estimates position and heading byscoring a dense floor grid through ray-cast object visibility, reaching amedian error of 0.95 m; and (iii) resolves residual ambiguity through aBayesian dialog module that asks targeted yes/no questions and updatesa pose posterior until the location is pinpointed. To support this task wecontribute a benchmark of 13,000+ pose-indexed natural-language de-scriptions over 1,300+ indoor 3D scans. Code and data will be released.Project page: https://rzninvo.github.io/Lang-Loc/.