PUF: Plug-and-Play Uncertainty-Aware Fusion for Online 3D Scene Graph Generation

Online 3D scene graph generation builds a persistent, struc-tured representation of a scene by incrementally fusing 2D observationsinto a global 3D graph. Existing online methods treat this fusion asa fully deterministic pipeline, where we identify three sources of un-certainty that are overlooked: observation, 2D model, and 3D repre-sentation. We propose PUF: a Plug-and-play, Uncertainty-aware, andtraining-free Fusion framework. Scene graph node association is refor-mulated as a probabilistic likelihood over semantic and spatial factors,replacing binary accept/reject gates. Dirichlet evidence accumulation dis-tributes class and relationship evidence across plausible candidates pro-portional to association likelihood. An optional class-conditional priorcompletes edges for sparsely or never co-observed object pairs. We instan-tiate PUF with both a 3D Gaussian and a 3D voxel backend and observeconsistent improvements, demonstrating its ability to generalize acrossdifferent representations. Experiments on the 3DSSG and ReplicaSSGbenchmarks show that our method substantially outperforms existingapproaches while maintaining real-time latency. These results establishuncertainty-aware fusion as a principled and effective paradigm for on-line 3D scene understanding. The source code is publicly available athttps://github.com/yyyyangyi/PUF.