Spotlights
CFM: Language-aligned Concept Foundation Model for Vision
Kai Wittenmayer ⋅ Sukrut Rao ⋅ Amin Parchami-Araghi ⋅ Bernt Schiele ⋅ Jonas Fischer
Language-aligned vision foundation models perform strongly across diverse downstream tasks. Yet, their learned representations remain opaque, making interpreting their decision-making difficult. Recent work decompose these representations into human-interpretable concepts, but provide poor spatial grounding and are limited to image classification tasks. In this work, we propose CFM, a language-aligned concept foundation model for vision that provides fine-grained concepts, which are human-interpretable and spatially grounded in the input image. When paired with a foundation model with strong semantic representations, we get explanations for any of its downstream tasks. Examining local co-occurrence dependencies of concepts allows us to define concept relationships through which we improve concept naming and obtain richer explanations. On benchmark data, we show that CFM provides performance on classification, segmentation, and captioning that is competitive with opaque foundation models while providing fine-grained, high quality concept-based explanations. Code at https://github.com/kawi19/CFM
Show more
GRF-Recon: Global Ray-Field Optimization for Long-Sequence Feed-forward Reconstruction
Enpeng Li ⋅ Yunzhou Zhang ⋅ Zhiyao Zhang ⋅ Dexuan Lyu ⋅ Chenyu Wang ⋅ Chiyuan Cui ⋅ Cheng Cheng
Feed-forward 3D reconstruction provides an efficient paradigmfor scene modeling from image sequences. Scaling these models to largemonocular scenarios are constrained by excessive GPU memory foot-print, degraded local geometry, and long-term trajectory drift. Exist-ing chunk-based optimization strategies provide limited geometric con-straints and fail to maintain global consistency over extended trajecto-ries. We present a unified framework for stable and scalable feed-forward3D reconstruction from long monocular sequences. Our approach buildson coarse-to-fine trajectory alignment augmented by lightweight geomet-ric prior injection. Distilling monocular geometric cues into the feed-forward backbone via LoRA adaptation improves depth accuracy on finestructures while preserving inference efficiency. We introduce a hybrid-weight sparse ray-field optimization that leverages high-frequency geo-metric features to guide local point-cloud refinement and enforce con-sistent inter-frame ray constraints. Unlike prior chunk-based methods,this establishes strong cross-frame geometric coupling while maintainingscalability. Finally, an efficient trajectory stitching strategy with jointray-error optimization explicitly reduces accumulated drift. Extensiveexperiments show that our approach achieves competitive trajectory ac-curacy compared with representative SLAM systems, while maintainingglobally consistent 3D reconstruction in large-scale scenarios.
Show more
Wat3R: Underwater 3D Geometry Learning without Underwater Annotations
Jiangwei Ren ⋅ Xingyu Jiang ⋅ Zijie Song ⋅ Wei Xu ⋅ Hongkai Lin ⋅ Dingkang Liang ⋅ Xiang Bai
Estimating 3D geometry in underwater environments presentsunique challenges due to light attenuation, scattering, and the absenceof large-scale, high-quality 3D annotations. Pioneering methods rely onmassive dense annotations that are impractical in underwater settings.In this paper, we propose Wat3R, a cross-domain semi-supervised learn-ing framework designed to adapt feed-forward 3D reconstruction modelsfrom air to underwater scenes. Uniquely, our method eliminates the needfor any annotated underwater data following a teacher-student architec-ture, that learns robust geometry representations merely on abundantunlabeled real underwater video footage. We also design a cross-viewconsistency loss that leverages geometric cues from other views to com-pensate for the information degradation in the current view caused bywater attenuation and scattering. Furthermore, considering the lack ofcomprehensive evaluation benchmarks, we construct Water3D, a di-verse dataset covering various water bodies and underwater scenarios, de-signed for geometric task evaluation. Experimental results demonstratethat Wat3R outperforms current state-of-the-art methods in underwatermulti-view depth estimation and point cloud reconstruction. The datasetand code are available at https://github.com/LSXI7/Wat3R .
Show more
AdaBoosting Text Prompts for Vision-Language Models
Seokhee Jin ⋅ Changhwan Sung ⋅ Sunung Mun ⋅ Hoyoung Kim ⋅ Jungseul Ok
The classification accuracy of pretrained Vision-Language Models (VLMs) relies on the quality of the text prompts. Handcrafted templates and Large Language Model (LLM)-generated descriptions not only make predictions more interpretable, but also enable reuse of the same prompts across heterogeneous VLMs. Recent works construct taskadapted text prompts with a small number of labeled images. However, existing few-shot text prompting methods do not explicitly focus on misclassified examples during prompt construction, leading to only marginal improvements even as more shots become available. To fully exploit few-shot supervision, we propose Text Prompt Boosting (TPB), an AdaBoost-inspired framework that treats each text-prompt-based classifier as a weak learner and sequentially aggregates them into a strong ensemble by explicitly targeting hard, misclassified examples. Extensive experiments show that TPB preserves task-intrinsic, model-agnostic cues in text space, enabling robust cross-model transfer. Across eleven classification benchmarks, TPB improves accuracy on the source model and preserves shot-driven gains when transferred to larger, more capable VLMs, where existing methods struggle to sustain such improvements.
Show more
GeoNVS: Geometry Grounded Video Diffusion for Novel View Synthesis
Minjun Kang ⋅ Inkyu Shin ⋅ Taeyeop Lee ⋅ Myungchul Kim ⋅ In Kweon ⋅ KUK-JIN YOON
Novel view synthesis requires strong 3D geometric consis-tency and the ability to generate visually coherent images across di-verse viewpoints. While recent camera-controlled video diffusion modelsshow promising results, they often suffer from geometric distortions andlimited camera controllability. To overcome these challenges, we intro-duce GeoNVS, a geometry-grounded novel-view synthesizer that en-hances both geometric fidelity and camera controllability through ex-plicit 3D geometric guidance. Our key innovation is the Gaussian Splat-ting Feature Adapter (GS-Adapter), which lifts input-view diffusion fea-tures into 3D Gaussian representations, renders geometry-constrainednovel-view features, and adaptively fuses them with diffusion features tocorrect geometrically inconsistent representations. Unlike prior methodsthat inject geometry at the input level, GS-Adapter operates in featurespace, avoiding view-dependent color noise that degrades structural con-sistency. Its plug-and-play design enables zero-shot compatibility withdiverse feed-forward geometry models without additional training, andcan be adapted to other video diffusion backbones. Experiments across 9scenes and 18 settings demonstrate state-of-the-art performance, achiev-ing 11.3% and 14.9% improvements over SEVA and CameraCtrl, withup to 2× reduction in translation error and 7× in Chamfer Distance.
Show more
Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrations
Chancharik Mitra ⋅ Yusen Luo ⋅ Raj Saravanan ⋅ Dantong Niu ⋅ Anirudh Pai ⋅ Jesse Thomason ⋅ Trevor Darrell ⋅ Abrar Anwar ⋅ Deva Ramanan ⋅ Roei Herzig
Vision-Language Action (VLAs) models promise to extendthe remarkable success of vision-language models (VLMs) to robotics. Yet,unlike VLMs in the vision-language domain, VLAs for robotics requirefinetuning to contend with varying physical factors like robot embodiment,environment characteristics, and spatial relationships of each task. Wepropose leveraging few-shot demonstrations of tasks to better capture thisphysical variability. Overfitting to the trained task is yet another problemof existing methods. Inspired by functional specificity, we hypothesizethat finetuning only task-specific sparse model representations is bothmore effective and better retains pretrained model generality. In thiswork, we introduce Robotic Steering, a finetuning approach groundedin mechanistic interpretability that leverages few-shot demonstrations toidentify and selectively finetune task-specific attention heads aligned withthe physical, visual, and linguistic requirements of robotic tasks. Throughcomprehensive on-robot evaluations with a Franka Emika robot arm, wedemonstrate that Robotic Steering outperforms LoRA while achievingsuperior robustness under task variation, reduced computational cost,and enhanced interpretability for adapting VLAs to diverse robotic tasks.
Show more
Molmo-Point: Better Pointing for VLMs with Grounding Tokens
Christopher Clark ⋅ Yue Yang ⋅ Jae Sung Park ⋅ Zixian Ma ⋅ Jieyu Zhang ⋅ Rohun Tripathi ⋅ Mohammadreza Salehi ⋅ Sangho Lee ⋅ Ranjay Krishna
Grounding has become a fundamental capability of vision-language models (VLMs). Most existing VLMs point by generating coor-dinates as part of their text output, which requires learning a complicatedcoordinate system and results in a high token count. Instead, we pro-pose a more intuitive pointing mechanism that directly selects the visualtokens that contain the target concept. Our model generates a specialpointing token that cross-attends to the input image or video tokens andselects the appropriate one. To make this model more fine-grained, we fol-low these pointing tokens with an additional special token that selects afine-grained subpatch within the initially selected region, and then a thirdtoken that specifies a location within that subpatch. We further showthat performance improves by generating points sequentially in a consis-tent order, encoding the relative position of the previously selected point,and including a special no-more-points class when selecting visual to-kens. Using this method, we set a new state-of-the-art on image pointing(70.7% on PointBench), set a new state-of-the-art for fully open modelson GUI pointing (61.1% on ScreenSpotPro), substantially improve VLMvideo tracking (62.5 on J &F vs 56.7 for Molmo2 on Molmo2Track), andimprove video pointing (59.1% human preference win rate vs. Molmo2).We additionally show that our method significantly improves learningefficiency and discuss the qualitative differences that emerge with thisdesign change. Our model weights, new datasets, and source code areavailable at https://allenai.org/blog/molmopoint.
Show more
DreamWorld: Geometry-Grounded Video Diffusion for 3D-Consistent World Modeling
Haibo Yang ⋅ Yang Chen ⋅ Yingwei Pan ⋅ Zhineng Chen ⋅ Ting Yao ⋅ Tao Mei
Camera-controlled video diffusion models (VDMs) have re-cently emerged as powerful world models, enabling users to explore 3Dscenes through flexible, user-defined camera trajectories. Nevertheless,current VDMs typically rely on implicit spatiotemporal representationswithout explicit 3D geometric grounding. Such geometry-agnostic mod-eling often leads to issues including geometrically implausible struc-tures and cross-view spatial inconsistencies. To alleviate this, we presentDreamWorld, a new recipe of world model that novelly bridges the strongspatial structure priors of 3D foundation models with the high-fidelitygenerative capabilities of video diffusion models for geometry-consistent3D scene generation. Specifically, given the input image and camera tra-jectory, DreamWorld first learns a geometry video diffusion model topredict compact geometry features for the target novel views, function-ing as explicit structure pivots to reflect the underlying 3D spatial layout.To achieve this, we introduce a distillation paradigm that transfers high-level structural knowledge from a pretrained 3D foundation model to thediffusion model, thereby enabling it to produce geometrically consistentand spatially coherent features. Conditioned on such geometry features,another appearance video diffusion model is then utilized to synthesizethe final video, ensuring improved geometric plausibility and cross-viewconsistency while maintaining high visual fidelity. Extensive experimentsdemonstrate that DreamWorld outperforms existing methods in visualquality, 3D consistency, and camera controllability. Our project page isavailable at https://yanghb22-fdu.github.io/DreamWorld.
Show more
Why Far Looks Up: Probing Spatial Representation in Vision-Language Models
Cheolhong Min ⋅ Jaeyun Jung ⋅ Daeun Lee ⋅ Hyeonseong Jeon ⋅ Yu Su ⋅ Jonathan Tremblay ⋅ Chan Hee Song ⋅ Jaesik Park
Vision-language models (VLMs) achieve strong performanceon spatial reasoning benchmarks, yet it remains unclear whether this re-flects structured 3D understanding or reliance on statistical shortcuts innatural images. We introduce a representation-level analysis frameworkthat constructs minimal contrastive pairs to measure how spatial axes areorganized and disentangled within VLM embeddings. Our analysis acrossmultiple model families reveals a consistent vertical-distance entangle-ment: models conflate vertical image position with distance, mirroringthe perspective bias of natural photographs. This bias produces a signif-icant accuracy gap between perspective-consistent and counter-heuristicexamples, and intensifies under data scaling even as overall benchmarkaccuracy improves. We further show that models with similar benchmarkscores can exhibit different internal representations, and that these dif-ferences predict accuracy and robustness across diverse spatial reasoningbenchmarks. To isolate this bias from evaluation-set skew, we introduceSpatialTunnel, a synthetic benchmark designed to expose spatial short-cut biases by removing common correlations present in natural images.Experiments suggest that the entanglement is model-intrinsic, and thatmodels with well-separated spatial axes exhibit greater robustness, indi-cating that well-structured spatial representations lead to more reliablespatial reasoning across diverse benchmarks. Code and benchmark areavailable on the project website.
Show more
Edit3r: Instant 3D Scene Editing from Sparse Unposed Images
Jiageng Liu ⋅ Weijie Lyu ⋅ Xueting Li ⋅ Yejie Guo ⋅ Ming-Hsuan Yang
We present Edit3r , a feed-forward framework that recon-structs and edits a 3D scene in a single pass. Unlike prior approachesthat require per-scene optimization, Edit3r directly predicts instruction-aligned 3D Gaussians from unposed sparse-view images and a text prompt.A key challenge in training 3D editing models is the lack of view-consistentedited images for supervision, due to the stochastic nature of 2D imageediting. We address this by reformulating editing as a recoloring pretexttask, and show that a model trained on simple recoloring transfers effec-tively to 3D-consistent editing at inference time. Specifically, we develop(i) a SAM2-based recoloring pipeline that generates one recolored ref-erence view along with reliable, cross-view-consistent supervision views,and (ii) an asymmetric input strategy that pairs the recolored refer-ence view with raw auxiliary views, encouraging the network to fuse andalign disparate observations. At inference, it supports diverse 2D editorsby resolving cross-view conflicts and yielding coherent 3D edits, withoutrequiring multi-view edited supervision during training. For large-scalequantitative evaluation, we introduce DL3DV-Edit-Bench. Compre-hensive quantitative and qualitative results show that Edit3r achievessuperior semantic alignment and enhanced 3D consistency compared torecent baselines, while operating at significantly higher inference speed,making it promising for real-time 3D editing applications.
Show more
Do Not Leave a Gap: Hallucination-Free Object Concealment in Vision-Language Models
Amira Guesmi ⋅ Muhammad Shafique
Vision-language models (VLMs) have recently shown remark-able capabilities in visual understanding and generation, but remainvulnerable to adversarial manipulations of visual content. Prior object-hiding attacks primarily rely on suppressing or blocking region-specificrepresentations, often creating semantic gaps that inadvertently inducehallucination, where models invent plausible but incorrect objects. In thiswork, we demonstrate that hallucination arises not from object absenceper se, but from semantic discontinuity introduced by such suppression-based attacks. We propose a new class of background-consistent objectconcealment attacks, which hide target objects by re-encoding their vi-sual representations to be statistically and semantically consistent withsurrounding background regions. Crucially, our approach preserves tokenstructure and attention flow, avoiding representational voids that trig-ger hallucination. We present a pixel-level optimization framework thatenforces background-consistent re-encoding across multiple transformerlayers while preserving global scene semantics. Extensive experimentson state-of-the-art vision-language models show that our method effec-tively conceals target objects while preserving up to 86% of non-targetobjects and reducing grounded hallucination by up to 3× compared toattention-suppression-based attacks. Qualitative results further confirmthat our approach maintains scene coherence and avoids spurious ob-ject insertion. Our findings highlight semantic continuity as a key factorin hallucination behavior and introduce a new direction for adversarialanalysis of generative multimodal models.
Show more
PriSplat: Propagating Reliable Multi-view Information for Distractor-Free 3DGS
Yunseo Yang ⋅ Youngho Yoon ⋅ KUK-JIN YOON
Applying 3D Gaussian Splatting (3DGS) to uncontrolled, in-the-wild environments remains challenging due to transient distractors.Existing approaches typically mask these distractors and suppress theirlosses during training. However, this zero-masking strategy leaves suchregions unsupervised, leading to per-view overfitting and causing severefloaters in novel views. To resolve this, it is essential to re-establish densemulti-view constraints by recovering the missing background informa-tion. In light of this, we propose PriSplat, a novel framework designed topropagate reliable multi-view information to restore these missing regionswith high geometric integrity. Specifically, we repurpose a large-scaleview synthesis prior into a 3D-aware inpainting engine, adapted throughmask-aware fast-weight updates to prevent distractor leakage into thescene memory. To ensure the fidelity of this restoration, we introducea geometry-aware support view selection algorithm based on informa-tion density and spatio-angular constraints. Ultimately, these synergisticadvancements yield 3D-consistent pseudo-ground truth from masked re-gions, establishing the dense supervision necessary to eliminate artifacts.Extensive experiments show that our method outperforms state-of-the-art baselines in both synthesis quality and multi-view consistency. Thecode is available at https://github.com/yun-seo/PriSplat.git.
Show more
Learning to Deny: Action Denial in Multimodal Large Language Models
Raiyaan Abdullah ⋅ Shehreen Azad ⋅ Yogesh Rawat
Multimodal large language models (MLLMs) have rapidlyadvanced video understanding, achieving strong zero-shot and few-shotrecognition across standard benchmarks. Yet their ability to deny anaction by recognizing when an activity is not happening despite strongcontextual cues remains largely unexplored. We introduce UCF101-AD, a large-scale benchmark consisting of paired Action-Presence andAction-Denial clips, designed to evaluate this capacity for denial. Eachnegative video in UCF101-AD preserves the same contextual and mo-tion cues (persons, objects, locations) as its positive counterpart, butthe defining action itself is explicitly absent. Evaluating 20 state-of-the-art MLLMs reveals a consistent failure: models that exceed 85% accu-racy on the positive action classes collapse below 50% on its action-denial counterpart, indicating a strong inclination to affirm plausibleactions rather than verify that they truly occur. This exposes a criti-cal blind spot in modern video understanding: the inability to reasoncausally about whether a motion actually happens. To probe this is-sue, we explore a causal graph formulation, CausalAct, which expressesscene structure through natural-language prompts linking context, in-teraction, and motion. Incorporating such causal cues substantially re-duces false positives, demonstrating that denial is a learnable reason-ing skill. UCF101-AD provides a new lens for diagnosing and improv-ing causal reasoning in multimodal models. Dataset and relevant code:https://github.com/raiyaan-abdullah/Learn-to-Deny.
Show more
ReSplat: Learning Recurrent Gaussian Splatting
Haofei Xu ⋅ Daniel Barath ⋅ Andreas Geiger ⋅ Marc Pollefeys
While existing feed-forward Gaussian splatting models of-fer computational efficiency and can generalize to sparse view settings,their performance is fundamentally constrained by relying on a singleforward pass for inference. We propose ReSplat, a feed-forward recurrentGaussian splatting model that iteratively refines 3D Gaussians withoutexplicitly computing gradients. Our key insight is that the Gaussiansplatting rendering error serves as a rich feedback signal, guiding therecurrent network to learn effective Gaussian updates. This feedbacksignal naturally adapts to unseen data distributions at test time, enablingrobust generalization across datasets, view counts, and image resolutions.To initialize the recurrent process, we introduce a compact reconstructionmodel that operates in a 16× subsampled space, producing 16× fewerGaussians than previous per-pixel Gaussian models. This substantiallyreduces computational overhead and allows for efficient Gaussian updates.Extensive experiments across varying number of input views (2, 8, 16, 32),resolutions (256×256 to 540×960), and datasets (DL3DV, RealEstate10K,and ACID) demonstrate that our method achieves state-of-the-art perfor-mance while significantly reducing the number of Gaussians and improvingthe rendering speed. Our project page is at haofeixu.github.io/resplat.
Show more
SA-ResGS: Self-Augmented Residual 3D Gaussian Splatting for Next Best View Selection
Kim Jun-Seong ⋅ Tae-Hyun Oh ⋅ Eduardo Pérez Pellitero ⋅ Youngkyoon Jang
We propose Self-Augmented Residual 3D Gaussian Splat-ting, a novel framework for stabilizing uncertainty quantification andenhancing uncertainty-aware supervision in Next-Best-View selection foractive scene reconstruction. To efficiently estimate scene coverage, SA-ResGS generates geometry-consistent Self-Augmented point clouds (SA-Points) via triangulation between observed training views and rasterizedextrapolated views. To address the lack of learning signals in underrep-resented regions within sparse, wide-baseline settings, we introduce thefirst skip-connection-inspired residual learning strategy tailored for 3DGS.This mechanism amplifies gradient flow to weakly contributing, high-uncertainty Gaussians. Our contributions are threefold: (1) a physicallygrounded, diversified view selection strategy; (2) an uncertainty-awareresidual supervision scheme that improves gradient flow and learningstability; and (3) implicitly debiased uncertainty quantification resultingfrom constrained view selection and residual supervision. Experiments onNeRF Synthetic, Mip-NeRF 360, and challenging extended benchmarkfrom Deep Blending and Tanks and Temples demonstrate that SA-ResGSconsistently outperforms state-of-the-art competing methods in bothreconstruction quality and view selection robustness.
Show more
PuzLM: Solving Jigsaw Puzzles with Sequence-to-Sequence Language Models
Gur Elkin ⋅ Ofir I Shahar ⋅ Ohad Ben-Shahar
Square jigsaw puzzles are typically solved by visually match-ing piece images to recover the original layout. This work introducesPuzLM, an alternative perspective that recasts jigsaw reassembly asa discrete sequence-to-sequence (Seq2Seq) problem, inspired by naturallanguage representations. We design an efficient puzzle quantization pro-cedure that transforms each piece into a short sequence of discrete tokens,enabling the direct application of standard Seq2Seq language models aspowerful jigsaw solvers. Our approach demonstrates that accurate puzzlereconstruction can be achieved through purely symbolic reasoning overdiscrete representations, improving state-of-the-art performance even onpuzzles with eroded boundaries or missing pieces.
Show more
GaussianLens: Localized High-Resolution Reconstruction via On-Demand Gaussian Densification
Yijia Weng ⋅ Zhicheng Wang ⋅ Songyou Peng ⋅ Saining Xie ⋅ Howard Zhou ⋅ Leonidas Guibas
We perceive our surrounding environments with an active fo-cus, paying more attention to regions of interest, such as the shelf labelsin a grocery store or a family photo on the wall. When it comes to scenereconstruction, this human perception trait calls for spatially varyingdegrees of detail ready for closer inspection in critical regions, prefer-ably reconstructed on demand as users shift their focus. While recentapproaches in 3D Gaussian Splatting (3DGS) can achieve fast, general-izable scene reconstruction from sparse views, their uniform resolutionoutput leads to high computational costs, making them unscalable tohigh-resolution training. As a result, they cannot leverage available imagecaptures at their original high resolution for detail reconstruction. Per-scene optimization methods reconstruct finer details with heuristic-basedadaptive density control, yet require dense observations and lengthy of-fline optimization. To bridge the gap between the prohibitive cost of high-resolution holistic reconstructions and the user needs for localized finedetails, we propose the problem of localized high-resolution reconstruc-tion through on-demand generalizable Gaussian densification. Given aninitial low-resolution 3DGS reconstruction, the goal is to learn a gener-alizable network that densifies the reconstruction to capture fine detailsin a user-specified local region of interest (RoI), based on sparse high-resolution observations of the RoI. This formulation avoids the high costand redundancy of uniformly high-resolution reconstructions and enablesthe full leverage of high-resolution observations in critical regions. To ad-dress the problem, we propose GaussianLens, a feed-forward densifica-tion framework that fuses multi-modal information from the initial 3DGSand multi-view images. We further propose a pixel-guided densificationmechanism that effectively captures details under significant resolutionincreases. Experiments demonstrate our method’s superior performancein local high-fidelity detail reconstruction and strong scalability to im-ages of up to 1024 × 1024 resolution.
Show more
GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing
Mingxin Liu ⋅ Ziqian Fan ⋅ Zhaokai Wang ⋅ Leyao Gu ⋅ Zirun Zhu ⋅ YiguoHe YiguoHe ⋅ Yuchen Yang ⋅ Changyao Tian ⋅ Xiangyu Zhao ⋅ Ning Liao ⋅ Shaofeng Zhang ⋅ Qibing Ren ⋅ Zhihang Zhong ⋅ Xuanhe Zhou ⋅ Junchi Yan ⋅ Xue Yang
Unified multimodal models target joint understanding, rea-soning, and generation, but current image editing benchmarks are largelyconfined to natural images and shallow commonsense reasoning, offeringlimited assessment of this capability under structured, domain-specificconstraints. In this work, we introduce GRADE, the first benchmarkto assess discipline-informed knowledge and reasoning in image editing.GRADE comprises 520 carefully curated samples across 10 academicdomains, spanning from natural science to social science. To supportrigorous evaluation, we propose a multi-dimensional evaluation proto-col that jointly assesses Discipline Reasoning, Visual Consistency, andLogical Readability. Extensive experiments on 20 state-of-the-art open-source and closed-source models reveal substantial limitations in currentmodels under implicit, knowledge-intensive editing settings, leading tolarge performance gaps. Beyond quantitative scores, we conduct rigor-ous analyses and ablations to expose model shortcomings and identifythe constraints within disciplinary editing. Together, GRADE pinpointskey directions for the future development of unified multimodal models,advancing the research on discipline-informed image editing and reason-ing. Our benchmark and evaluation code are publicly released.
Show more
On Test-Time Scaling for Vision-Language Models
Fawaz Sammani ⋅ Tzoulio Chamiti ⋅ Nikos Deligiannis
Test-time scaling is a paradigm where large models use ad-ditional compute at inference to achieve better performance, withoutchanging model weights. While it has been widely studied for Large Lan-guage Models (LLMs), its applicability to Large Vision-Language Mod-els (LVLMs) remains less explored and analyzed, with limited analysis ofwhether, when, and to what extent these approaches transfer to LVLMs.In this work, we ask a simple but fundamental question: can conven-tional test-time scaling methods developed for LLMs be directly appliedto LVLMs? We present the first comprehensive study of test-time scal-ing for LVLMs, spanning multiple models and model sizes, nine test-timescaling methods, and six diverse benchmarks. Our main findings is that1) different from previous findings, small, well-performing models benefitthe most from test-time scaling, enabling performance improvements ofup to around 30%, reaching large models performance, and often out-performing them, 2) LVLMs lose focus when given more compute thannecessary, and 3) Visual information is encoded early in the reasoningchain, after which the chain is dominated by text-only reasoning andthe contribution of image tokens drops significantly. Finally, we also pro-vide a global and fine-grained analysis on the quality and informationsufficiency of the reasoning chains produced. Overall, our findings andanalysis provide practical guidance and insights into LVLMs and theirdeployment in research and industry.
Show more
SubSplat: High-Resolution Pixel-aligned 3DGS via Sub-pixel Gaussian Reparameterization
Jiun Lee ⋅ Jaekwang Kim ⋅ Sangmin Lee
Pixel-aligned Gaussian splatting enables efficient and gen-eralizable novel-view synthesis. However, high-resolution rendering facesa critical trade-off where increasing input resolution improves detail atthe expense of quadratically rising network computational cost. Con-versely, maintaining low-resolution inputs stabilizes this cost but resultsin insufficient Gaussian density and artifacts. To address this, we proposeSubSplat, which introduces Sub-pixel Gaussian Reparameterizer(SPGR)to subdivide primary Gaussians into fine-grained primitives, restoringstructural density directly from low-resolution features. We further en-hance the reparameterization quality through feature aggregation, whicheffectively captures high-frequency details across multiple views. Experi-ments on RealEstate10K and ACID demonstrate that SubSplat achieveshigh-fidelity rendering with superior efficiency. Our results validate thatthe proposed framework successfully resolves the trade-off between repa-rameterization fidelity and network computational cost inherent in pixel-aligned Gaussian Splatting.
Show more
Deformable Triangle Splatting: Flexible Primitives for Real-Time Radiance Field Rendering
Oriol Jiménez-Ayguadé ⋅ Antonio Agudo
Recent radiance field methods represent scenes with 2D prim-itives that offer surface alignment and efficient rasterization, from Gaus-sian disks to triangles, yet all rely on convex boundaries: curved and con-cave structures demand excessive primitives. We introduce DeformableTriangle Splatting, which augments each triangle with K control pointsper edge, each parameterized by a single learnable scalar displacementthat shifts the boundary inward or outward, enabling non-convex shaperepresentation while preserving the three base vertices that define the 3Dplane. To render these non-convex primitives differentiably, we designa rasterization pipeline in the triangle’s barycentric coordinate space,ensuring view-consistent rendering. A winding number test determineswhether each pixel lies inside the deformed primitive, and a windowfunction controlled by two learnable parameters, sharpness and cornersmoothness, together with a per-primitive scalar opacity, produces thesmooth opacity transition from interior to boundary. Validation is donein a variety of real-world scenes, outperforming recent works based onnon-volumetric primitives in terms of visual quality and versatility whilestill achieving competitive rendering efficiency.Project page: orioljim1.github.io/detris
Show more
URoPE: Universal Relative Position Embedding across Geometric Spaces
Yichen Xie ⋅ Depu Meng ⋅ Yihan Hu ⋅ Chensheng Peng ⋅ Quentin HERAU ⋅ Masayoshi TOMIZUKA ⋅ Wei Zhan
Relative position embedding has become a standard mech-anism for encoding positional information in Transformers. However,existing formulations are typically limited to a fixed geometric space,namely 1D sequences or regular 2D/3D grids, which restricts their ap-plicability to many computer vision tasks that require geometric rea-soning across camera views or between 2D and 3D spaces. To addressthis limitation, we propose URoPE, a universal extension of RotaryPosition Embedding (RoPE) to cross-view or cross-dimensional geomet-ric spaces. For each key/value image patch, URoPE samples 3D pointsalong the corresponding camera ray at predefined depth anchors andprojects them into the query image plane. Standard 2D RoPE can thenbe applied using the projected pixel coordinates. URoPE is a parameter-free and intrinsics-aware relative position embedding that is invariant tothe choice of global coordinate systems, while remaining fully compatiblewith existing RoPE-optimized attention kernels. We evaluate URoPE asa plug-in positional encoding for transformer architectures across a di-verse set of tasks, including novel view synthesis, 3D object detection,object tracking, and depth estimation, covering 2D–2D, 2D–3D, and tem-poral scenarios. Experiments show that URoPE consistently improvesthe performance of transformer-based models across all tasks, demon-strating its effectiveness and generality for geometric reasoning. Our codeis available on our project website: https://urope-pe.github.io/.
Show more
Fourier Compressor: Frequency-Domain Visual Token Compression for Vision-Language Models
Huanyu Wang ⋅ Jushi Kai ⋅ Haoli Bai ⋅ Lu Hou ⋅ Bo Jiang ⋅ Ziwei He ⋅ Zhouhan Lin
Vision-Language Models (VLMs) incur substantial computational overhead and inference latency due to the large number of visual tokens introduced by high-resolution image and video inputs. Existing parameter-free token compression methods typically rely on token selection or merging, yet they risk discarding substantial visual information or distorting the original representation distribution, resulting in pronounced performance degradation at high compression ratios. In response, we aim to explore a more effective and efficient visual token compression strategy, with a promising direction in the frequency domain. Motivated by the success of frequency-domain transforms in image compression (e.g., JPEG), we systematically analyze the frequency redundancy in visual representations and uncover a nonuniform distribution of semantic information across frequency bands. Building upon this, we introduce Fourier Compressor, an effective, parameter-free, and highly generalizable module that removes redundancy from visual representations within the frequency domain. Implemented via FFT with O(n2 logn) complexity and no additional parameters, Fourier Compressor introduces negligible computational overhead while preserving semantic fidelity. Extensive experiments on imagebased benchmarks demonstrate that our method achieves a favorable performance-efficiency trade-off, retaining over 96% of the original accuracy while reducing inference FLOPs by up to 83.8% and boosting generation speed by 31.2%. It consistently outperforms existing parameterfree methods and even surpasses some parameterized approaches. Importantly, Fourier Compressor generalizes consistently across both LLaVA and Qwen-VL architectures, and further extends to video understanding tasks, highlighting its practical applicability for efficient VLMs.
Show more
Neural Harmonic Textures for High-Quality Primitive Based Neural Reconstruction
Jorge Condor ⋅ Nicolas Moënne-Loccoz ⋅ Merlin Nimier-David ⋅ Piotr Didyk ⋅ Zan Gojcic ⋅ Qi Wu
Primitive-based methods such as 3D Gaussian Splatting haverecently become the state-of-the-art for novel-view synthesis and relatedreconstruction tasks. Compared to neural fields, these representationsare more flexible, adaptive, and scale better to large scenes. However,the limited expressivity of individual primitives makes modeling high-frequency detail challenging. We introduce Neural Harmonic Textures, aneural representation approach that anchors latent feature vectors on avirtual scaffold surrounding each primitive. These features are interpo-lated within the primitive at ray intersection points. Inspired by Fourieranalysis, we apply periodic activations to the interpolated features, turn-ing alpha blending into a weighted sum of harmonic components. Theresulting signal is then decoded in a single deferred pass using a smallneural network, significantly reducing computational cost. Neural Har-monic Textures yield state-of-the-art results in real-time novel view syn-thesis while bridging the gap between primitive- and neural-field-basedreconstruction. Our method integrates seamlessly into existing primitive-based pipelines such as 3DGUT, Triangle Splatting, and 2DGS. We fur-ther demonstrate its generality with applications to 2D image fitting andsemantic reconstruction.
Show more
EmbedCopilot: Evaluating Vision-Language Models for Hardware-Aware Embedded System Development
Dongsheng Yuan ⋅ Yimo Deng ⋅ Huangxun Chen
We introduce EmbedCopilot-Bench, a multimodal bench-mark designed to evaluate large vision-language models (LVLMs) as as-sistants for embedded development. Compared with pure software pro-gramming, embedded programming inherently involves hardware, mak-ing it well-suited to benefit from LVLMs’ visual capabilities. However, tothe best of our knowledge, no existing benchmark comprehensively eval-uates how effectively LVLMs can serve as embedded copilots. Our bench-mark is developed to fill this gap. Built on real-world embedded develop-ment videos, we constructed 216 annotated multimodal QA triplets span-ning heterogeneous hardware platforms and peripheral modules, cov-ering hardware operation, software configuration, and code generationtasks. To assess model performance, we pair a rubric-driven LLM-as-a-Judge protocol with execution-centric evaluation in Wokwi platformand hardware-in-the-loop Execution Success Rate (ESR), jointly cap-turing semantic quality and functional correctness. Experiments on arange of closed- and open-source LVLMs show that visual context sig-nificantly boosts performance, especially for hardware-related tasks, butthat models remain far from reliable embedded copilots. ESR analysesand a pin-localization case study reveal frequent near-miss failures causedby brittle grounding and limited structured reasoning over board layouts.EmbedCopilot-Bench provides a challenging, realistic testbed for futuremethods that more tightly integrate perception, circuit understanding,and executable code synthesis for real-world embedded development.
Show more
CubicSplat: Differentiable Vector Graphics via Error-Bounded Forward Relaxation
Chenglong Liu ⋅ Xin Zhang ⋅ Yimeng Zhu ⋅ Liyang He ⋅ Yixiao Ma ⋅ Yu Su ⋅ Zhenya Huang ⋅ Qi Liu
Vector graphics are prized for their resolution independence,compact storage, and direct editability, making differentiable optimiza-tion of their parametric primitives an attractive goal. Yet classical ras-terization is discontinuous with respect to geometry, and existing reme-dies that smooth the forward pass demand increasingly elaborate heuris-tics as scene complexity grows. We trace this fragility to a gradientseesaw: design choices that improve forward geometric exactness cansystematically degrade the induced gradient signal, and vice versa. Tonavigate this tension we introduce CubicSplat, a differentiable vectorrasterizer that replaces Bézier closest-point solvers with uniform poly-line surrogates whose geometric error is bounded at O(S −2 ). The re-sulting static computation graph yields well-conditioned gradients byconstruction, while a compositing-derived visibility mechanism prunesdegenerate primitives without auxiliary regularization. On DIV2K andKodak benchmarks CubicSplat achieves state-of-the-art reconstructionquality with over 2 dB PSNR gain in the closed-fill setting, while train-ing up to 4× faster than prior methods. The code is available at https://github.com/CubicSplat/repo
Show more
SOCO: Benchmarking Semantic Object Correspondence in Vision Foundation Models
Olaf Dünkel ⋅ Basavaraj Sunagad ⋅ Haoran Wang ⋅ David Hoffmann ⋅ Christian Theobalt ⋅ Adam Kortylewski
Measuring structured object understanding in vision founda-tion models remains challenging due to inconsistent evaluation protocolsand limited part-level supervision. Semantic correspondence (SC) eval-uates this capability by testing whether object parts can be matchedacross instances and categories under large variations in appearance,viewpoint, and geometry. To enable a systematic SC evaluation, we in-troduce SOCO, a new benchmark for Semantic Object Correspondencethat introduces a taxonomy of correspondence types and provides con-sistent, functionally meaningful keypoint annotations across 100 cate-gories and over 1M correspondence pairs. In addition, SOCO includeskeypoint language descriptions, enabling the evaluation of large vision-language models (LVLMs) and their fine-grained part-level understand-ing. Comprehensive experiments reveal that (i) vision foundation back-bones encode strong semantic structure but transfer correspondencespoorly across related categories and only partially capture object-partposition, (ii) LVLMs are stronger at text-prompted part localizationthan at visual-reference cross-image matching, exposing a gap betweenlanguage-grounded localization and fine-grained visual correspondence,and (iii) correspondence performance predicts dense downstream tasks—segmentation, tracking, 3D pose estimation, and 3D detection—morestrongly than ImageNet classification. Together, these findings positionSOCO as a benchmark for structured, part-level representation qualityin vision and multimodal foundation models.
Show more
RayRoPE: Projective Ray Positional Encoding for Multi-view Attention
Yu Wu ⋅ Minsik Jeon ⋅ Rick Chang ⋅ Oncel Tuzel ⋅ Shubham Tulsiani
We study positional encodings for multi-view transformersthat process tokens from a set of posed input images, and seek a mech-anism that encodes patches uniquely, allows SE(3)-invariant attentionwith multi-frequency similarity, and can adapt to the geometry of theunderlying 3D scene. We find that prior (absolute or relative) encod-ing schemes for multi-view attention do not meet these desiderata, andpresent RayRoPE to address this gap. RayRoPE represents patch po-sitions based on associated rays and computes query-frame projectivecoordinates to ensure SE(3) invariance. To adapt to scene geometry,RayRoPE predicts (without direct supervision) a per-token depth to ob-tain its position along the corresponding ray, while also modeling uncer-tainty and analytically computing the expected positional encoding. Wevalidate our method on the tasks of novel-view synthesis, stereo depthestimation, and feed-forward 3DGS reconstruction. While remaining ef-ficient, RayRoPE consistently improves over alternate position encodingschemes (e.g. 24% relative improvement on LPIPS in RE10K).
Show more
TetraSDF: Analytic Isosurface Extraction with Multi-resolution Tetrahedral Grid
Seonghun Oh ⋅ Youngjung Uh ⋅ Jin-Hwa Kim
Extracting an explicit surface that exactly matches the zerolevel set of a neural signed distance function (SDF) remains challenging. Sampling-based isosurfacing methods such as Marching Cubes introduce discretization error. In contrast, continuous piecewise affine (CPWA) analytic approaches typically require plain ReLU MLPs, which limits the ability to learn high-frequency SDFs in practice. We present TetraSDF, an analytic isosurface extraction framework for SDFs that retains the expressiveness of grid-based encoders while enabling exact zero-level set extraction, by representing the SDF with a ReLU MLP composed with a multi-resolution tetrahedral positional encoder. Our positional encoder’s barycentric interpolation preserves a global CPWA structure, allowing us to track ReLU linear regions within an encoder-induced polyhedral complex. We further introduce a fixed analytic input preconditioner derived from the encoder’s metric to reduce directional bias, thereby stabilizing training. Across multiple benchmarks, TetraSDF matches or surpasses existing grid-based encoders in SDF reconstruction accuracy, while faithfully recovering the network’s zero-level set as a triangle mesh.
Show more
On the Reliability of Cue Conflict and Beyond
Pum Jun Kim ⋅ Seung-Ah Lee ⋅ Seongho Park ⋅ Dongyoon Han ⋅ Jaejun Yoo
Understanding how neural networks rely on visual cues offers a human-interpretable view of their internal decision processes. The cueconflict benchmark has been influential in probing shape-texture preference and in motivating the insight that stronger, human-like shape bias is often associated with improved in-domain performance. However, we find that the current stylization-based instantiation can yield unstable and ambiguous bias estimates. Specifically, stylization may not reliably instantiate perceptually valid and separable cues nor control their relative informativeness, ratio-based bias can obscure absolute cue sensitivity, and restricting evaluation to preselected classes can distort model predictions by ignoring the full decision space. Together, these factors can confound preference with cue validity, cue balance, and recognizability artifacts. We introduce REFINED-BIAS, an integrated dataset and evaluation framework for reliable and interpretable shape–texture bias diagnosis. REFINED-BIAS constructs balanced, humanand modelrecognizable cue pairs using explicit definitions of shape and texture, and measures cuespecific sensitivity over the full label space via a ranking-based metric, enabling fairer cross-model comparisons. Across diverse training regimes and architectures, REFINED-BIAS enables fairer cross-model comparison, more faithful diagnosis of shape and texture biases, and clearer empirical conclusions, resolving inconsistencies that prior cue-conflict evaluations could not reliably disambiguate. Our code is available at REFINED-BIAS.
Show more
Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation
Ziyu Zhang ⋅ Yi Yu ⋅ Simeng Zhu ⋅ Ahmed Aly ⋅ Yunhe Gao ⋅ Ning Gu ⋅ Yuan Xue
Accurate anatomical structure segmentation in medical im-age is essential for diagnosis and treatment planning. While recent inter-active segmentation foundation models enhance generalization throughlarge-scale multimodal pretraining, they still depend on precise promptsand can fail in underrepresented clinical contexts (e.g., small organs-at-risk). We present AtlasSegFM, an atlas-guided framework that cus-tomizes off-the-shelf foundation models to new clinical contexts with asingle annotated example. AtlasSegFM 1) performs atlas-query regis-tration to generate context-aware prompts, 2) refines the segmentationwith a frozen foundation model, and 3) applies a lightweight adaptivefusion module to combine atlas priors with foundation-model inputs andpredictions. Extensive experiments on six public and in-house datasetsacross radiotherapy and vascular scenarios show consistent gains, withthe largest improvements on small and delicate structures. AtlasSegFMprovides a lightweight, deployable solution for one-shot customization ofsegmentation foundation models in real-world clinical workflows.
Show more
One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models
Xiaohao Xu ⋅ Feng Xue ⋅ Xiang Li ⋅ Haowei Li ⋅ Shusheng Yang ⋅ Tianyi Zhang ⋅ Matthew Johnson-Roberson ⋅ Xiaonan Huang
A faithful 3D world representation should account for layeredgeometry, where a single camera ray may contain multiple visible and ge-ometrically valid surfaces. Monocular depth estimation, however, reducesthis structure to one scalar depth per pixel. Transparent scenes makethis ambiguity measurable: the same ray can pass through foregroundglass and observe the background, turning the supervised target into aconvention of annotation, data, and training rather than a scene-intrinsictruth. A learned predictor exposes this convention as its depth-layerpreference. We introduce MultiDepth-3k (MD-3k), a sparse two-layer or-dinal benchmark for measuring depth-layer preference and multi-layerspatial relationship accuracy (ML-SRA). On MD-3k, leading depth foun-dation models exhibit diverse layer preferences under standard RGBinput, showing that the same layered geometry can be resolved differentlyacross models. We further find that Laplacian Visual Prompting (LVP),a training-free spectral input transformation, can substantially changethe reported layer for certain frozen models. The strongest RGB/LVPpair, DAv2-L, reaches 75.5% ML-SRA. These results suggest that depthfoundation models may express complementary geometric hypotheses thatstandard RGB inference leaves unexpressed. We invite the community torethink depth supervision and evaluation through an ambiguity-awarelens, where multiple valid 3D interpretations are treated as geometricstructure to be measured, preserved, and expressed.
Show more
Mimicking Radiologists: A Coarse-to-Fine Framework with Structural Sparse Tokens for Dual-LLM Computed Tomography Report Generation
Hong Liu ⋅ Dong Wei ⋅ Yefeng Zheng ⋅ Xian Wu ⋅ Liansheng Wang
Computed tomography report generation (CTRG) automatesradiology reporting to reduce clinical workload and facilitate patient care.Recent efforts in applying the rapidly developing large language mod-els (LLMs) have advanced the field; yet, they still face a fundamentalchallenge due to the large volume of 3D data: effectively reducing highfeature redundancy and computational burden while simultaneously ex-tracting information-rich representations. To address this challenge, thiswork presents a novel CTRG framework that fully imitates the coarse-to-fine visual search pipeline practiced by radiologists. Our frameworkfirst employs a ViT-based image-text alignment architecture to extracta global token and local patch tokens for each anatomical structure, en-hanced by a mask-guided sparse negative-entropy loss. An abnormality-proposal LLM then processes the global token and clinical metadata topropose a shortlist of candidate abnormalities. This shortlist promptsan abnormality-prompted local token filter (AP-LTF) to select the mostinformative patch tokens, effectively reducing redundancy while preserv-ing critical information. Finally, a report-generation LLM takes in theglobal token, selected local tokens, and clinical metadata to composea full report. During training, ground-truth abnormalities are prefixedto the reference report to enhance awareness of clinic-relevant findings,and group relative policy optimization (GRPO) aligns the abnormality-proposal and report-generation LLMs for collaborative efficacy. Exper-imental results on two public CTRG datasets demonstrate the supe-rior performance of our framework compared to existing state-of-the-artFig. 1: Comparison of token compression strategies for CTRG. (a) 3D spatial pooling[2, 8] achieves high compression rate but incurs significant information loss. (b) Mask-guided volume cropping [12, 22] yields region-specific tokens but omits intra-regionredundancy. (c) [26] selects the top K tokens with the largest attention weights toregion-specific queries, yet may be subject to dominant “sink tokens” [14, 29]. (d) Ourmethod first proposes a shortlist of candidate abnormalities using only a single globaltoken per structure and clinical metadata. Then, an abnormality-prompted local tokenfilter (AP-LTF) combines the shortlist, a learned score, and the attention weight for acomprehensive token selection, thereby preventing sink tokens from dominating.methods in terms of clinical efficacy, RaTEScore, and GREEN scores.Ablation studies further validate the effectiveness of its novel design.
Show more
VesselTok: Tokenizing Vessel-like 3D Biomedical Graph Representations for Reconstruction and Generation
Chinmay Prabhakar ⋅ Bastian Wittmann ⋅ Tamaz Amiranashvili ⋅ Paul Büschl ⋅ Ezequiel De la Rosa ⋅ Julian McGinnis ⋅ Benedikt Wiestler ⋅ Bjoern Menze ⋅ Suprosanna Shit
Spatial graphs provide a lightweight and elegant represen-tation of curvilinear anatomical structures such as blood vessels, lungairways, and neuronal networks. Accurately modeling these graphs iscrucial in clinical and (bio-)medical research. However, the high spatialresolution of large networks drastically increases their complexity, result-ing in significant computational challenges. In this work, we aim to tacklethese challenges by proposing VesselTok, a framework that approachesspatially dense graphs from a parametric shape perspective to learn la-tent representations (tokens). VesselTok leverages centerline points witha pseudo radius to effectively encode tubular geometry. Specifically, welearn a novel latent representation conditioned on centerline points to en-code neural implicit representations of vessel-like, tubular structures. Wedemonstrate VesselTok’s performance across diverse anatomies, includ-ing lung airways, lung vessels, and brain vessels, highlighting its ability torobustly encode complex topologies. To prove the effectiveness of Vessel-Tok’s learned latent representations, we show that they (i) generalize tounseen anatomies, (ii) support generative modeling of plausible anatom-ical graphs, and (iii) transfer effectively to downstream inverse problems,such as link prediction.
Show more
TR-MoE: Temporal Reliability-Aware Mixture-of-Experts for Robust Tracking
Tianle Wang ⋅ Xiangyang Yang ⋅ Jihua Zhu ⋅ Binrui Liu ⋅ Yanzhao Li ⋅ Shuiwang Li
In visual object tracking, SAM2 stands out in deformation adaptability and distinguishing similar objects due to its pixel-level masks, whereas traditional discriminative trackers have advantages under occlusion and motion blur thanks to global semantic feature matching. To leverage this complementarity, we propose TR-MoE, a Temporal Reliability-aware Mixture-of-Experts framework. Unlike conventional fusion strategies that rely on static or per-frame confidence, we reframe tracker fusion as a time-series modeling problem. Specifically, we introduce a Mamba-based Trajectory Refiner (MTR) that processes the historical sequence of tracker states—including relative deviations and consistency scores—to capture the temporal evolution of error regimes. This enables the model to adaptively allocate fusion weights among experts based on historical temporal context. Furthermore, to achieve precise geometric correction, we design a KAN-enhanced Disentangled Head. Leveraging Kolmogorov–Arnold Networks, this module factorizes the refinement into independent directional offsets, dynamic scale magnitudes, and fusion weights. Extensive experiments show that TR-MoE effectively adapts to varying failure patterns, significantly boosting robustness and establishing new state-of-the-art performance. The source code is available at https://github.com/Avant1v/TR-MoE.
Show more
RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement
Jiyao Wang ⋅ Qingyong Hu ⋅ Duoxun Tang ⋅ Xiao Yang ⋅ Kaishun Wu ⋅ Jiangbo Yu
Video-based remote physiological measurement (RPM) ishighly accessible but remains fragile under varying illumination, skintones, and motion. Radio frequency (RF) radar is largely invariant to il-lumination and appearance, providing complementary cardio-respiratorymicro-motion cues; however, requiring radar at inference is often imprac-tical due to its limited ubiquity and deployment overhead. We proposeRPM-Distill, a physiology-guided cross-modal distillation frameworkthat leverages synchronized radar only during training while retainingvideo-only inference. Our key observation is that although RGB and RFwaveforms differ in sensing physics and time-domain morphology, theyshare similar latent periodic rhythm in the frequency domain. We thusdistill physiology-structured spectral evidence to improve robustness, vialosses that (i) anchor the fundamental peak, (ii) match the off-peak back-ground distribution, and (iii) preserve spectral morphology and sharp-ness. To avoid negative transfer under sample-level teacher quality andalignment uncertainty, a spectral policy network predicts sample-leveldistillation gates and component weights from the student–teacher spec-tral relation map, learned with a meta bilevel objective on a small labeledvalidation split. Through extensive experiments in challenging conditionsand cross-dataset settings, RPM-Distill reduces MAE by up to 81% andimproves correlation by up to 21% over the best video-only baseline.Code is at https://github.com/WJULYW/RPM-Distill.
Show more
Generating Multi-view Adversarial Examples for Visual Geometry Grounded Transformer
Qi Song ⋅ Ziyuan Luo ⋅ Haoliang Han ⋅ Renjie Wan
The Visual Geometry Grounded Transformer (VGGT) enables unified feed-forward 3D reconstruction from multi-view images. However, deploying such a high-performance model may expose critical security vulnerabilities. Traditional adversarial perturbations require costly per-scene optimization, while Universal Adversarial Perturbations (UAPs) rely on a single static pattern and fail to effectively attack VGGT. To address these limitations, we propose MVAP-G, a multiview adversarial perturbation generator that produces imperceptible consistent perturbations across multiple views in a single feed-forward pass. To ensure perturbation consistency across diverse scenes, we design a cross-view adversarial alignment mechanism to process multi-view images. Experiments demonstrate that MVAP-G significantly degrades VGGT performance without iterative optimization during inference. This work pioneers multi-view adversarial attacks on 3D foundation models, uncovering severe vulnerabilities and underscoring the urgent need for robust 3D vision systems. The code is available at https://github.com/ qsong2001/mvap-g.
Show more
Pushing the Limits of High-Resolution Weather Forecasting through Data Scaling
Yang Zhao ⋅ Peisong Niu ⋅ Tian Zhou ⋅ Ziqing Ma ⋅ Guanlong Ma ⋅ Rong Jin ⋅ Huiling Yuan ⋅ Liang Sun
The development of 0.1◦ global weather forecasting modelsbased on machine learning (ML) is constrained by the limited availabil-ity of high-resolution data, as decades of reanalysis are only availableat 0.25◦ resolution. While existing approaches fine-tune 0.25◦ forecastmodels on limited 0.1◦ samples, we show that this transfer is hinderedby the irreversible information loss inherent in coarse-resolution fore-casting. Therefore, we propose BaguanHR, a framework that shiftsthe focus from transferring models to transferring data. We first showthat super-resolution (SR) has lower conditional entropy and input am-plification than forecasting, making it a more robust vehicle for reso-lution transfer. By leveraging this advantage through variable-wise SR,we synthesize extensive 0.1◦ data from ERA5. BaguanHR’s performanceon the synthetic-plus-real dataset exceeds both ML-based methods andIFS-HRES, achieving superior performance across over 85% of the leadtimes within 72 hours. Furthermore, our findings highlight a power-lawscaling effect, as a twofold increase in data reduces RMSE by 4.6% for 72-hour forecasting and 4.9% for 120-hour forecasting. Our results demon-strate that scaling high-resolution ML-based forecasting is primarily adata bottleneck, and that variable-wise super-resolution provides a sim-ple yet general solution to unlock long coarse-resolution reanalyses forhigh-resolution training.
Show more
LoRC: Detecting AI-Generated Images via Low-Rank Collapse in the Semantic-Residual Space
Haozhen Yan ⋅ Ruoxin Chen ⋅ Jiahui Zhan ⋅ Taiping Yao ⋅ Bo Wang ⋅ Youchang xiao ⋅ Shouhong Ding ⋅ Liqing Zhang ⋅ Jianfu Zhang
Modern generators faithfully model macroscopic semantics,producing synthetic images that appear highly realistic. Consequently,decisive forensic cues reside in subtle non-semantic visual discrepancies.To reveal these cues, we revisit AIGI detection from a geometric perspec-tive and identify an architecture-agnostic signature. Specifically, mod-ern generators exhibit low-rank collapse (i.e., rank degeneracy) in thesemantic-residual orthogonal subspace while largely preserving the dom-inant semantic direction. This structural flattening consistently emergesduring the final decoding stage, forming a shared bottleneck across di-verse generator architectures. Motivated by this signature, we proposeLoRC, a framework that decouples semantic dominance to capture thecollapsed residual geometry induced by the generative decoding bottle-neck. Our method improves accuracy by an average of 7.0% across mul-tiple benchmarks and achieves 97.0% accuracy on 39 unseen generators.These results demonstrate strong cross-model generalization and robust-ness, making LoRC a reliable approach for AIGI detection in complexreal-world environments.
Show more
Ice Cloud Geometry Retrieval with Calibrated Uncertainty from Passive Satellite Imagery
Ayush Prasad
We present a dense prediction system for ice cloud geom-etry retrieval from extremely sparse supervision, where active satellitesensors (EarthCARE radar/lidar) provide accurate but spatially narrowtraining labels (∼1.6% of pixels) and passive imagers (VIIRS, VisibleInfrared Imaging Radiometer Suite) observe the full globe. A ConvNex-tUNet trained on co-located tracks predicts eight targets at every pixelwith calibrated 90% prediction intervals using conformalized quantileregression (CQR). The best configuration, a five-member quantile en-semble, achieves R2= 0.742 with prediction quality constant regardlessof distance to the supervision track, confirming that the model learnsa per-pixel spectral retrieval rather than interpolating from nearby la-bels. Calibration is robust across latitudes and cloud types, with deepconvective clouds as the main failure mode. Deployed globally, the sys-tem produces 609M predictions from one day of VIIRS data in 2.4 hourson a single GPU, enabling dense 3D cloud characterization at planetaryscale.1
Show more
Proteus: Model Leakage-Induced Adversarial Attack in Federated Learning
Junjie Shan ⋅ Yue Zhang ⋅ Ziqi Zhao ⋅ Ka Ho Chow
The massive volume and privacy-sensitive nature of visualdata have made federated learning (FL) a preferred paradigm for train-ing vision models across distributed data sources. However, during train-ing, FL repeatedly shares the in-progress model with randomly selectedparticipants. This paper investigates an overlooked yet practical threatarising from this sharing process: leaked intermediate models can be ex-ploited by adversaries to craft adversarial examples that compromise thefinal deployed model. Although directly using an intermediate model,especially one leaked early in training, as a surrogate yields only mod-erate attack gains, it can serve as an anchor for anticipating subsequenttraining dynamics. Based on this insight, we propose Proteus, a modelleakage-induced adversarial attack that leverages a leaked model to iden-tify vulnerabilities that persist throughout training, thereby generatingadversarial examples that remain effective against the final deployedmodel. For the first time, we show that models exposed well before con-vergence can already pose substantial risks to the final model, even if itundergoes hundreds of additional training rounds after leakage. Exten-sive experiments across diverse datasets, neural architectures, and FLconfigurations confirm the severity of this threat. Proteus exploits in-herent model leakage in FL and improves the attack success rate from59.75% when directly using the leaked model to 85.40%, even when leak-age occurs after only 30% of the total training process.
Show more
Beyond Filter Pruning: Top-K Spatial Selection for Efficient Neural Networks
Sarthak Ketanbhai Modi ⋅ Hans Soegeng ⋅ Thomas Peyrin
Deep neural networks have shown remarkable performance across diverse domains, but their substantial model size and computational requirements hinder deployment in memoryand computationconstrained environments. Despite progress in pruning, existing methods primarily target inter-filter redundancy and leave significant spatial redundancy within convolutional kernels unaddressed. The resulting models retain many correlated spatial patterns, limiting achievable efficiency gains. We propose Top-K Pruning (TKP), a pruning framework that extends sparsity to the spatial dimension by retaining only the most informative positions within each convolutional kernel. TKP introduces a two-stage procedure: an auxiliary regularization phase that concentrates information into dominant spatial locations, followed by deterministic Top-K selection that yields semi-structured sparsity. This transforms dense convolutions into efficient selective-sampling operations with strictly bounded computational cost. Across diverse architectures, including CNNs, vision transformers, vision-language models, and a large language model, TKP consistently achieves strong accuracy–efficiency trade-offs. On CIFAR and ImageNet models, TKP matches or exceeds the accuracy of prior structured pruning methods while delivering up to 8.9× theoretical FLOP reduction. TKP remains robust under quantization-aware training, achieving 16–17× compression with minimal accuracy loss on ResNet–18 and VGG–19. Moreover, TKP generalizes to BLIP-Base and LLaMA-2-7B, outperforming state-of-the-art pruning baselines. These results highlight TKP as a simple and effective approach for removing spatial redundancy in modern vision and largescale models.
Show more
Event-based Sparse-view Background-Oriented Schlieren Tomography
Xinyu Zhou ⋅ Shihao Hu ⋅ Peiqi Duan ⋅ Chao Xu ⋅ Boxin Shi
Background-oriented schlieren (BOS) tomography reconstructs3D flow density fields from refraction-induced distortions observed fromone or more viewpoints. Capturing high-speed flows with frame-basedBOS typically requires high-speed cameras and bright illumination tocompensate for short exposure times. In this paper, we propose an event-based BOS tomography approach that reconstructs time-varying 4D den-sity fields from event streams. Leveraging the high temporal resolutionand high dynamic range of event cameras, the proposed approach enableshigh-speed airflow reconstruction under ambient lighting. We representthe spatiotemporal density field as a neural implicit field and render BOSobservations via refractive ray tracing. We utilize physics-informed regu-larization to improve reconstruction under sparse views. Experiments onsimulated and real data in single-view and orthogonal dual-view setupsdemonstrate accurate reconstructions.
Show more
Spectral Gating via Damped Oscillations for Adaptive Implicit Neural Representations
Alex Costanzino ⋅ Pierluigi Zama Ramirez ⋅ Giuseppe Lisanti ⋅ Luigi Di Stefano
Implicit Neural Representations (INRs) have been provensuccessful in encoding continuous signals through coordinate-based net-works, yet facing a spectral dilemma: periodic activations capture finedetails but act as all-pass filters that memorise noise, while spatiallycompact activations regularise effectively but suffer from low-frequencybias. Existing attempts to resolve this trade-off introduce computationaloverhead or tuning frailty. We propose to model each neuron’s activationas the steady-state response of a sinusoidally-forced damped harmonicoscillator, whose amplitude naturally governs the network’s spectral se-lectivity during training. By jointly optimising the oscillator parametersalongside the network weights, our method adapts to the target signal’sspectral content without explicit regularisation. Initialised in the stop-band, the network exhibits a coarse-to-fine learning curriculum that pro-gressively expands its spectral gate, capturing low-frequency structuresfirst and high-frequency details only when justified by the reconstructionobjective. Comprehensive experiments show that our approach consis-tently achieves state-of-the-art or competitive results against establishedINRs, while requiring no task-specific tuning of any hyperparameters.Project Page available at https://alex-costanzino.github.io/fdho/.
Show more
Zero-shot Depth from Defocus
Yiming Zuo ⋅ Hongyu Wen ⋅ Venkat Subramanian ⋅ Patrick Chen ⋅ Karhan Kayan ⋅ Mario Bijelic ⋅ Felix Heide ⋅ Jia Deng
Depth from Defocus (DfD) is the task of estimating a densemetric depth map from a focus stack. Unlike previous works overfittingto a certain dataset, this paper focuses on the challenging and practi-cal setting of zero-shot generalization. We first propose a new real-worldDfD benchmark ZEDD, which contains 8.3× more scenes and signifi-cantly higher quality images and ground-truth depth maps comparedto previous benchmarks. We also design a novel network architecturenamed FOSSA. FOSSA is a Transformer-based architecture with noveldesigns tailored to the DfD task. The key contribution is a stack atten-tion layer with a focus distance embedding, allowing efficient informa-tion exchange across the focus stack. Finally, we develop a new trainingdata pipeline allowing us to utilize existing large-scale RGBD datasetsto generate synthetic focus stacks. Experiment results on ZEDD andother benchmarks show a significant improvement over the baselines,reducing errors by up to 55.7%. The ZEDD benchmark is released athttps://zedd.cs.princeton.edu. The code and checkpoints are re-leased at https://github.com/princeton-vl/FOSSA.Input: Focus stack and corresponding focus distancesFocus Distance = 0.8m Focus Distance = 1.6m Focus Distance = 3.1m Focus Distance = 8.1mDFF-FV 𝛿! = 0.71 MoGe-2 𝛿! = 0.45 𝛿! = 0.98Existing depth from defocus Monocular depth models suffer Ours Ground Truthmodels are not generalizable from scale ambiguity*Equal contribution.
Show more
Bayesian Self-Attention with Local Pixel Correlations for Lightweight Denoising Transformers
Runyang He ⋅ Zuowei Shen ⋅ Hui JI
Transformer architectures have achieved strong performancein image denoising, but their computational and memory costs remainhigh. We revisit self-attention (SA) from a Bayesian perspective and showthat standard SA mainly exploits first-order inter-patch statistics, resem-bling a learnable non-local averaging scheme. However, this formulationneglects second-order intra-patch statistics, which are important for cap-turing local pixel dependencies. To address this, we introduce a BayesianSA formulation that jointly models first-order inter-patch and second-order intra-patch statistics. This leads to a lightweight denoising Trans-former, termed NLformer, featuring a dual-branch attention design andan efficient feed-forward module. Experiments on several benchmarksshow that NLformer outperforms existing lightweight denoising networksand substantially narrows the gap to full-size Transformer-based denois-ers while maintaining low model complexity.
Show more
When Higher Order Hurts: Pre-Asymptotic Order Collapse in Generative ODE Sampling — A Theory of Discretization–Learning Interaction
Farzad Salajegheh ⋅ Sudhir Mudur
Higher-order ODE solvers are widely believed to improvesample quality in diffusion and flow-based generative models at a fixednumber of function evaluations (NFE). We show that this assumptioncan fail: at practical step sizes, higher-order methods can systematicallyunderperform first-order Euler. The mechanism is a multiplicative in-teraction between discretization and learning error. Applying backwarderror analysis to a learned field fˆ = f +ε reveals interaction terms hk ∇k εthat dominate classical truncation error at practical NFEs in diffusionand flow models, causing a solver of nominal order p to behave as lowerorder—pre-asymptotic order collapse. Our analysis yields a predictivecrossover step size h⋆ characterizing when higher order helps or hurts.Validation on toy manifolds and pretrained diffusion and flow modelsconfirms the predicted hierarchy: Euler is best at low NFE, Heun over-takes at moderate budgets, and RK4 can surpass Heun at higher NFE.Building on this theory, we propose an adaptive sampler that dynami-cally selects solver order based on the local roughness score R = h/h⋆ ,estimated at zero extra NFE, and often improves sample quality acrossmodels and datasets.
Show more
Structured-Noise Masked Modeling for Video, Audio and Beyond
Aritra Bhowmik ⋅ Carlos Hinojosa ⋅ Fida Mohammad Thoker ⋅ Bernard Ghanem ⋅ Cees Snoek
Masked modeling has emerged as a robust self-supervisedlearning framework. However, most methods rely on random masking,which disregards the structural properties of different data modalities. Toalign with the spatiotemporal and spectral characteristics of video andaudio data, we introduce a structured noise-based masking approach.By filtering white noise into different color noise distributions, we gen-erate structured masks that capture modality-specific patterns withoutrequiring handcrafted heuristics or access to the data. Our approach en-hances masked video and audio modeling frameworks without any addi-tional computational cost. Experiments show that structured noise mask-ing consistently outperforms random masking, underscoring the value ofmodality-aware masking strategies for representation learning.
Show more
PolarAPP: Beyond Polarization Demosaicking for Polarimetric Applications
Yidong Luo ⋅ Chenggong Li ⋅ Yunfeng Song ⋅ Ping Wang ⋅ Boxin Shi ⋅ Junchao Zhang ⋅ Xin Yuan
Polarimetric imaging enables advanced vision applicationssuch as normal estimation and de-reflection by capturing unique surface-material interactions. However, existing applications (alternatively calleddownstream tasks) rely on datasets constructed by naïvely regroupingraw measurements from division-of-focal-plane sensors—where pixels ofthe same polarization angle are extracted and aligned into sparse im-ages without proper demosaicking. This reconstruction strategy resultsin suboptimal, incomplete targets that limit downstream performance.Moreover, current demosaicking methods are task-agnostic, optimizingonly for photometric fidelity rather than utility in downstream tasks.Towards this end, we propose PolarAPP, the first framework to jointlyoptimize demosaicking and its downstream tasks. PolarAPP introducesa feature alignment mechanism that semantically aligns the representa-tions of demosaicking and downstream networks via meta-learning, guid-ing the reconstruction to be task-aware. It further employs an equivalentimaging constraint for demosaicking training, enabling direct regressionto physically meaningful outputs without relying on rearranged data.Finally, a task-refinement stage fine-tunes the task network using thestable demosaicking front-end to further enhance accuracy. Extensiveexperimental results demonstrate that PolarAPP outperforms existingmethods in both demosaicking quality and downstream performance.Code is available here.
Show more
Stokes-Informed Diffusion for Robust Linear Polarization Estimation
Yidong Luo ⋅ Chenggong Li ⋅ Yuchao Feng ⋅ Boxin Shi ⋅ Junchao Zhang ⋅ Xin Yuan
Polarization cues benefit applications such as material de-tection and de-reflection, yet acquiring them typically requires dedicatedhardware. This motivates us to estimate the linear polarization from asingle RGB image. However, the task is inherently ill-posed, with theAngle of Polarization (AoP) becoming particularly unstable in weak-polarization regions, where the polarimetric signal is overwhelmed bynoise, leading to erratic angle estimates. To address these limitations, wepropose GenPolar, a Stokes-informed diffusion framework grounded inthe Mueller formalism from an intensity observation. Specifically, Gen-Polar predicts channel-wise linear Stokes components (S1 , S2 ) from inten-sity S0 , from which degree of linear polarization (DoLP) and AoP are an-alytically derived; AoP is further supervised with an observability-awareloss. In addition, to enable efficient and high-fidelity inference, we adopta two-stage training strategy. Firstly, a multi-step conditional diffusionmodel is trained with a physics-based loss. Subsequently, we distill it intoa one-step generator, which further supports stable Low-Rank Adapta-tion (LoRA) of the VAE encoder to mitigate domain-specific autoen-coding bias. Extensive experiments across rotating-polarizer, division-of-focal-plane, and hybrid datasets demonstrate that GenPolar achievesstate-of-the-art performance in both DoLP fidelity and AoP stability.Crucially, these improvements translate to significant and consistent gainsin downstream applications, including material detection and de-reflection.
Show more
Evaluating the Interpretability of Sparse Autoencoders with Concept Annotations
Jonas Klotz ⋅ Cassio F. Dantas ⋅ Pallavi Jain ⋅ Diego Marcos ⋅ Begüm Demir
Sparse autoencoders (SAEs) are increasingly used to extract interpretable concepts from vision and vision language models, yet existing evaluation methods largely rely on proxy metrics or qualitative inspection rather than measuring semantic correspondence. We present a human-grounded evaluation framework that quantifies alignment between SAE latents and human-annotated concepts, without requiring user studies, and validate this matching through targeted attribute perturbations. To enable this intervention-style evaluation in vision, we construct synCUB and synCOCO, synthetic benchmarks of paired images that differ in exactly one attribute. We introduce Fully-Binary Matching Pursuit (FBMP), a coalition-based matching procedure that supports many-to-one mappings between SAE latents and annotated concepts, and consistently outperforms one-to-one baselines. For functional validation, we propose a Targeted Attribute Perturbation Alignment Score (TAPAScore), which tests whether matched concepts respond selectively and in the expected direction under targeted image-level attribute perturbations. Under sanity checks, our matching and TAPAScore are the only evaluated metrics that reliably distinguish trained SAEs from untrained ones. Across SAEs trained on CLIP and DINOv2 embeddings, we find that increased overcompleteness can reduce perturbation alignment, indicating a reduction in interpretability. Our evaluation framework suggests that moderate dictionary sizes provide the best trade-off, yielding the most interpretable SAEs. Code and datasets are available at https://github.com/JonasKlotz/sae-concept-eval.
Show more
Off the Planckian Locus: Using 2D Chromaticity to Improve In-Camera Color
SaiKiran Tedla ⋅ Joshua Little ⋅ Hakki Karaimer ⋅ Michael S Brown
Traditional in-camera colorimetric mapping relies on corre-lated color temperature (CCT)–based interpolation between pre-calibratedtransforms optimized for Planckian illuminants such as CIE A and D65.However, modern lighting technologies such as LEDs can deviate sub-stantially from the Planckian locus, exposing the limitations of rely-ing on conventional one-dimensional CCT for illumination characteriza-tion. This paper demonstrates that transitioning from 1D CCT (on thePlanckian locus) to a 2D chromaticity space (off the Planckian locus) im-proves colorimetric accuracy across various mapping approaches. In addi-tion, we replace conventional CCT interpolation with a lightweight multi-layer perceptron (MLP) that leverages 2D chromaticity features for ro-bust colorimetric mapping under non-Planckian illuminants. A lightbox-based calibration procedure incorporating representative LED sourcesis used to train our MLP. Validated across diverse LED lighting, ourmethod reduces angular reproduction error by 22% on average in LED-lit scenes, maintains backward compatibility with traditional illuminants,accommodates multi-illuminant scenes, and supports real-time in-cameradeployment with negligible additional computational cost. Code and datacan be found on the project webpage: ccmmlp.github.io
Show more
Multi-Anchor Distillation with Text-Guided Analytic Classifier for Continual Learning
Qier Meng ⋅ De Cheng ⋅ Jiahao Li ⋅ Cheng Deng
Continual learning suffers from two forms of catastrophic forgetting: encoder drift, where visual representations shift as the model adapts to new tasks, and classifier overwriting, where gradient-based updates distort decision boundaries for previously learned classes. We present a unified exemplar-free framework that tackles both effects at the encoder and classifier levels. For the encoder, multi-anchor distillation maintains consistency with multiple historical encoder states and effectively bounds cumulative feature drift along the task sequence. For the classifier, a text-guided analytic classifier computes weights in closed form from incrementally accumulated statistics, completely bypassing gradient-based optimization. To further enhance the classifier, we incorporate a text prior that anchors classifier weights near CLIP’s text embeddings to inject cross-modal semantic knowledge, and propose text subspace regularization that encourages the classifier to learn directions orthogonal to the text embedding subspace, capturing discriminative visual patterns that text semantics alone cannot represent. Experiments on CIFAR-100, ImageNet-100, ImageNet-R, and CUB-200 show state-ofthe-art performance, including gains of +2.7% on CIFAR-100 and +3.5% on ImageNet-100 over prior methods. The code will be included in the supplementary material.
Show more
RoMa v2: Harder Better Faster Denser Feature Matching
Johan Edstedt ⋅ David Nordström ⋅ Yushan Zhang ⋅ Georg Bökman ⋅ Jonathan Astermark ⋅ Viktor Larsson ⋅ Anders Heyden ⋅ Fredrik Kahl ⋅ Mårten Wadenbäck ⋅ Michael Felsberg
Dense feature matching aims to estimate all correspondencesbetween two images of a 3D scene and has recently been established asthe gold standard due to its high accuracy and robustness. However,existing dense matchers still fail or perform poorly for many hard real-world scenarios, and high-precision models are often slow, limiting theirapplicability. In this paper, we attack these weaknesses on a wide frontthrough a series of systematic improvements that together yield a sig-nificantly better model. In particular, we construct a novel matchingarchitecture and loss, which, combined with a curated diverse trainingdistribution, enables our model to solve many complex matching tasks.We further make training faster through a decoupled two-stage matching-then-refinement pipeline, and at the same time, significantly reduce re-finement memory usage through a custom CUDA kernel. Finally, weleverage the recent DINOv3 foundation model along with multiple otherinsights to make the model more robust and unbiased. In our extensiveset of experiments, we show that the resulting novel matcher sets a newstate-of-the-art, being significantly more accurate than its predecessors.ω
Show more
Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length
Yubo Huang ⋅ Hailong Guo ⋅ Fangtai Wu ⋅ Weiqiang Wang ⋅ Shijie Huang ⋅ Qijun Gan ⋅ Shifeng Zhang ⋅ Lin Liu ⋅ Sirui Zhao ⋅ Enhong Chen ⋅ Jiaming Liu ⋅ Steven Hoi
Audio-driven avatar interaction demands real-time, stream-ing, and in_x001C_nite-length generation_x0016_capabilities fundamentally at oddswith the sequential denoising and long-horizon drift of current di_x001B_usionmodels. We present Live Avatar, an algorithm-system co-designed frame-work that addresses both challenges for a 14-billion-parameter di_x001B_usionmodel. On the algorithm side, a two-stage pipeline distills a pre-trained bidirectional model into a causal, few-step streaming one whoseKV cache stores noisy rather than clean history. This lossy condition-ing, counter-intuitively, suppresses error accumulation and enables mod-els trained on short clips to generalize stably beyond 10,000 seconds.On the system side, this noisy conditioning also breaks the sequen-tial sampling bottleneck, allowing Timestep-forcing Pipeline Parallelism(TPP) to assign each GPU a _x001C_xed denoising timestep, converting the se-quential di_x001B_usion chain into an asynchronous spatial pipeline that simul-taneously boosts throughput and improves temporal consistency. LiveAvatar achieves 45 FPS with a TTFF of 1.21 s on 5 H100 GPUs, andto our knowledge is the _x001C_rst to enable practical real-time streaming ofa 14B di_x001B_usion model for in_x001C_nite-length avatar generation. Our projectpage is at https://liveavatar.github.io/.
Show more
Mind-to-Face: Neural-Driven Photorealistic Avatar Synthesis via EEG Decoding
Haolin Xiong ⋅ Tianwen Fu ⋅ Yunxuan Cai ⋅ Pratusha Prasad ⋅ Haiwei Chen ⋅ Wenbin Teng ⋅ Hanyuan Xiao ⋅ Yajie Zhao
Current expressive avatar systems rely heavily on visual cuesand often fail when faces are occluded or emotions remain internal. Wepresent Mind-to-Face, the first framework to decode non-invasive elec-troencephalogram (EEG) signals directly into high-fidelity facial expres-sions. We build a dual-modality recording setup that captures synchro-nized EEG and multi-view facial video during emotion-eliciting stimuli,providing precise supervision for neural-to-visual learning. Our modeluses a CNN-Transformer encoder to map EEG signals into dense 3Dposition maps that sample over 65k vertices, capturing fine-scale geom-etry and subtle emotional dynamics, and renders them through a mod-ified 3D Gaussian Splatting pipeline for photorealistic, view-consistentresults. Extensive evaluations show that EEG alone can reliably predictdynamic, subject-specific facial expressions, including subtle emotionalresponses, demonstrating that neural signals contain far richer affectiveand geometric information than previously assumed. Mind-to-Face es-tablishes a new paradigm for neural-driven avatars, enabling personal-ized, emotion-aware telepresence and cognitive interaction in immersive
Show more
LoMa: Local Feature Matching Revisited
David Nordström ⋅ Johan Edstedt ⋅ Georg Bökman ⋅ Jonathan Astermark ⋅ Anders Heyden ⋅ Viktor Larsson ⋅ Mårten Wadenbäck ⋅ Michael Felsberg ⋅ Fredrik Kahl
Local feature matching has long been a fundamental component of 3D vision systems such as Structure-from-Motion (SfM), yet progress has lagged behind the rapid advances of modern data-driven approaches. The newer approaches, such as feed-forward reconstruction models, have benefited extensively from scaling dataset sizes, whereas local feature matching models are still only trained on a few mid-sized datasets. In this paper, we revisit local feature matching from a datadriven perspective. In our approach, which we call LoMa, we combine large and diverse data mixtures, modern training recipes, scaled model capacity, and scaled compute, resulting in remarkable gains in performance. Since current standard benchmarks mainly rely on collecting sparse views from successful 3D reconstructions, the evaluation of progress in feature matching has been limited to relatively easy image pairs. To address the resulting saturation of benchmarks, we collect 1000 highly challenging image pairs from internet data into a new dataset called HardMatch. Ground truth correspondences for HardMatch are obtained via manual annotation by the authors. In our extensive benchmarking suite, we find that LoMa makes outstanding progress across the board, outperforming the state-of-the-art method ALIKED+LightGlue by +18.6 mAA on HardMatch, +29.5 mAA on WxBS, +21.4 (1m, 10→) on InLoc, +24.2 AUC on RUBIK, and +12.4 mAA on IMC 2022. We release our code and models publicly at https://github.com/davnords/LoMa.
Show more
Gravity-aware partially calibrated absolute pose estimation from affine- or rotation-covariant features
Marcus Valtonen Örnhag ⋅ Alberto Jaenal Gálvez ⋅ Stefan Adalbjörnsson
Inertial measurement units (IMUs) are now standard in mostconsumer devices, such as smartphones, drones, and extended reality(XR) headsets. By fusing visual and inertial data, localization systemsgain significantly in speed and robustness compared to vision-only orIMU-only approaches. However, traditional pose estimation methods failto utilize the local geometric information embedded in feature descriptorslike SIFT. Recent work has proved the advantages of leveraging this in-formation for relative and absolute pose estimation, but its application topartially calibrated absolute pose estimation remains unexplored. In thispaper, we derive novel constraints for joint estimation of absolute poseand focal length, making use of a gravity vector obtained from IMU dataand the feature-induced local geometry, which we use to construct twoefficient solvers: UP1PfAC, that operates given a single affine correspon-dence and UP2PfORI, which requires two orientation-covariant features.Unlike traditional, semi-calibrated absolute pose methods requiring fourpoint correspondences, our solvers benefit from fewer samples and lowercomputational cost, simplifying robust estimation in modern RANSAC-like frameworks. We evaluate the proposed solvers against the state-of-the-art on large-scale public datasets and demonstrate that our methodachieves fast and accurate localization and focal length estimation.
Show more
GenLCA: 3D Diffusion for Full-Body Avatars from In-the-Wild Videos
Yiqian Wu ⋅ Rawal Khirodkar ⋅ Egor Zakharov ⋅ Timur Bagautdinov ⋅ Lei Xiao ⋅ Zhaoen Su ⋅ Shunsuke Saito ⋅ Xiaogang Jin ⋅ Junxuan Li
We present GenLCA, a diffusion-based generative modelfor generating and editing photorealistic full-body avatars from text andimage inputs. The generated avatars are faithful to the inputs, whilesupporting high-fidelity facial and full-body animations. The core ideais a novel paradigm that enables training a full-body 3D diffusion modelfrom partially observable 2D data, allowing the training dataset to scaleto millions of real-world videos. This scalability contributes to the supe-rior photorealism and generalizability of GenLCA. Specifically, we scaleup the dataset by repurposing a pretrained feed-forward avatar recon-struction model as an animatable 3D tokenizer, which encodes unstruc-tured video frames into structured 3D tokens. However, most real-worldvideos only provide partial observations of body parts, resulting in ex-cessive blurring or transparency artifacts in the 3D tokens. To addressthis, we propose a novel visibility-aware diffusion training strategy thatreplaces invalid regions with learnable tokens and computes losses onlyover valid regions. We then train a flow-based diffusion model on the to-ken dataset, inherently maintaining the photorealism and animatabilityprovided by the pretrained avatar reconstruction model. Our approacheffectively enables the use of large-scale real-world video data to traina diffusion model natively in 3D. We demonstrate the efficacy of ourmethod through diverse and high-fidelity generation and editing results,outperforming existing solutions by a large margin.
Show more
Rolling Shutter Camera Self-Calibration
Yongcong Zhang ⋅ Navid Rabbani ⋅ Bangyan Liao ⋅ Chengbo Wang ⋅ Yizhen Lao ⋅ Adrien Bartoli
Rolling shutter (RS) cameras are widely used in consumerdevices, but their row-wise exposure causes distortions under motion,making geometric 3D vision problems dependent on both camera in-trinsics and readout time ratio. Existing RS calibration methods relyon calibration targets or specialised hardware, limiting their use in un-constrained settings. We present the first self -calibration method forRS cameras that directly estimates camera intrinsics and the readouttime ratio from image sequences, without requiring calibration targets.The method is implemented as a self-calibrating bundle adjustment(BA), which critically depends on the RS imaging model. We com-bine two known complementary models. The first formulates RS imag-ing as continuous-time trajectory estimation under a row-wise pose rep-resentation. The second interprets RS images as temporally distortedglobal shutter (GS) images and requires to estimate correction fields.The combination is non-trivial and results in a unified dual-projectionmodel, in which each 3D point is simultaneously constrained at both row-dependent and reference timestamps along a shared continuous trajec-tory, enforcing stronger geometric and temporal consistency. Extensivesimulations analyse the applicability of several implementations undervarying conditions, and real data experiments demonstrate the accuracy,robustness, and practical effectiveness of the proposed approach.
Show more
OmniFace: Bridging the Image-to-Video Gap for High-Fidelity Face Swapping via Diffusion Transformer
Xu Guo ⋅ Fulong Ye ⋅ Xinghui Li ⋅ Pengqi Tu ⋅ Pengze Zhang ⋅ Qichao Sun ⋅ Songtao Zhao ⋅ Xiangwang Hou ⋅ Qian HE
Video Face Swapping (VFS) requires seamlessly injecting asource identity into a target video while meticulously preserving the orig-inal pose, expression, lighting, background, and dynamic information.Existing methods struggle to maintain identity similarity and attributepreservation while preserving temporal consistency. To address the chal-lenge, we propose a comprehensive framework to seamlessly transfer thesuperiority of Image Face Swapping (IFS) to the video domain. Wefirst introduce a novel data pipeline SyncID-Pipe that pre-trains anIdentity-Anchored Video Synthesizer and combines it with IFS modelsto construct bidirectional ID quadruplets for explicit supervision. Build-ing upon paired data, we propose a powerful Diffusion Transformer-basedframework OmniFace, employing a core Modality-Aware Conditioningmodule to discriminatively inject multi-model conditions. Meanwhile,we propose a Synthetic-to-Real Training mechanism and an Identity-Coherence Reward Weighting strategy to enhance visual realism andidentity consistency under challenging scenarios. To address the issue oflimited benchmarks, we introduce IDBench-V, a comprehensive bench-mark encompassing diverse scenes. Extensive experiments demonstrateOmniFace outperforms state-of-the-art methods and further exhibits ex-ceptional versatility, which can be seamlessly adapted to various swap-related tasks.
Show more
Lost in the Tail: Addressing Geographic Imbalance in Urban Visual Place Recognition
Zoey Shu ⋅ Jiacheng Yang ⋅ Yang Lu ⋅ Waishan Qiu ⋅ Chuan Li ⋅ Da Chen
Urban-scale Visual Place Recognition (VPR) aims to identify the geographic location of a query image by matching it against a geo-tagged database. While recent methods achieve impressive performance, they overlook a serious long-tailed problem hidden in urban-scale datasets, which biases the model towards locations with abundant images and ignores less-visited areas, causing models to systematically favor frequently photographed locations while failing in sparsely covered areas. In this paper, we systematically characterize this imbalance challenge and propose Distribution-Aware Place Recognition (DAPR), a model-agnostic plug-in framework that rebalances gradient contributions across head and tail classes. Additionally, within classification-retrieval pipelines, DAPR applies a multi-scale distance search mechanism to compute per-class distributional compactness, providing complementary gains at the retrieval stage. On the large-scale SF-XL benchmark, our framework outperforms the previous classification-retrieval baseline by 18.3% on test set v1, and 6.7% on test set v2. As a plug-in module, it achieves consistent improvements across representative VPR methods on SF-XL, MSLS, and Pitts30k, demonstrating broad generalizability across different methods and benchmarks.
Show more
OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Data
Kaixing Yang ⋅ Jiashu Zhu ⋅ Xulong Tang ⋅ Ziqiao Peng ⋅ Xiangyue Zhang ⋅ Chubin Chen ⋅ Puwei Wang ⋅ Jiahong Wu ⋅ Xiangxiang Chu ⋅ Hongyan Liu ⋅ Jun He
Music-driven dance video generation aims to synthesize expressive human motion that is temporally aligned with music while maintaining high visual fidelity. Despite recent progress, existing methods still struggle to generate dance videos that simultaneously exhibit expressive motion and high visual quality. This limitation primarily arises from two factors: (1) Dataset. The lack of large-scale and high-quality datasets and effective data collection pipelines specifically tailored for dance video generation; and (2) Method. The absence of principled framework-level solutions for effectively integrating music as a complementary conditioning signal into the Video Generation Foundation Models. To address the dataset limitation, we introduce CIPE-Dance, a large-scale Internetsourced dance video dataset, equipped with Choreograph Informed text annotations and constructed via a Progressive Expert pipeline. To the best of our knowledge, CIPE-Dance is the largest dataset for dance video generation to date, comprising 300k high-quality clips (over 400 hours) and covering diverse dancers, environments, and dance genres. To overcome the method limitation, we propose OmniDance, a frameworklevel recipe for integrating music into a TI2V foundation model without sacrificing its original controllability or visual fidelity. Motivated by the complementary roles of text (low-frequency semantics) and music (highfrequency temporal dynamics), OmniDance co-designs a depth-aware specialization model architecture, an anchored easy-to-hard curriculum learning strategy, and modality-specialized time-dependent CFG strategy, achieving unified TI2V/MI2V/MTI2V generation. Extensive experiments on the CIPE-Dance dataset demonstrate that OmniDance achieves state-of-the-art performance across TI2V, MI2V, and MTI2V tasks, while exhibiting robust multimodal integration capability. Project is available at https://github.com/AMAP-ML/OmniDance.
Show more
OpenCVL: An Open, Diverse, and Large-Scale Dataset for Fine-Grained Cross-View Localization
Zimin Xia ⋅ Mubariz Zaffar ⋅ Junsheng Fu ⋅ Alexandre ALahi ⋅ Julian Kooij
Fine-grained Cross-View Localization (CVL) estimates theprecise position and orientation of a ground-level image by aligningit with geo-referenced aerial imagery, offering a scalable alternative toGlobal Navigation Satellite Systems (GNSS) in challenging urban envi-ronments. Existing datasets rely on data collected with high-end sensorsuites, which inherently limit image diversity and scalability. While in-the-wild images are abundant, their noisy geo-tags make them unsuit-able for reliable evaluation. To bridge this gap, we introduce OpenCVL,a large-scale, diverse, and open dataset containing 617,388 ground-aerialimage pairs spanning 41 cities across four European countries. All imagesare sourced from permissive platforms, ensuring long-term accessibilityand supporting open and reproducible research. The training set com-bines images captured with high-end sensors with diverse in-the-wild im-agery. We further develop a data curation framework that filters and cor-rects pose annotations to construct reliable in-the-wild evaluation data.In addition, OpenCVL includes dedicated cross-area and snowy test setsto assess generalization and robustness. Experiments with a state-of-the-art CVL model on OpenCVL show that incorporating noisy in-the-wilddata consistently improves performance on clean test sets, suggesting apromising direction for scaling CVL with diverse real-world imagery.
Show more
MAVIN: Multi-Shot Audio-Visual Generation with Customized Narrative Control
Kaiqi Liu ⋅ Yunyao Mao ⋅ Ziqi Cai ⋅ Zheng Geng ⋅ Jing Wang ⋅ Qiulin Wang ⋅ Xintao Wang ⋅ Pengfei Wan ⋅ Kun Gai ⋅ Shuchen Weng ⋅ Boxin Shi
While recent generative models produce high-fidelity videos,they struggle with the complex narrative control required for coherentmulti-shot audio-visual generation. Existing methods suffer from tem-poral misalignment, limited controllability, and incomplete scripting. Inthis paper, we propose MAVIN, the first framework for multi-shot audio-visual generation with customized narrative control. To resolve tempo-ral misalignment, we propose boundary-aware attention, which lever-ages hierarchical captions and boundary-aware token routing to renderaudio-visual elements within their respective temporal boundaries. Toimprove the controllability for multi-subject scenarios, we propose ID-aware propagation, utilizing identity embeddings and an identity-awaremask to bind specific identities to consistent visual appearances and vocaltimbres. To provide comprehensive audio-visual narratives, we present amulti-agent scripting pipeline to transform free-form user inputs into hi-erarchical captions. Furthermore, we construct MAVINSet, a multi-shotaudio-visual dataset for robust training and evaluation. Extensive exper-iments demonstrate that MAVIN achieves state-of-the-art performance,opening up a new avenue for integrating generative models into profes-sional filmmaking workflows.
Show more
Progressive Pose-Guided 4D Animal Reconstruction from Monocular Video
Siyuan Li ⋅ Weiying Chen ⋅ Yilin Wang ⋅ Xinxin Zuo ⋅ Xingyu Li ⋅ Li Cheng
Reconstructing 4D animals from monocular videos is chal-lenging due to large inter-species variation, complex articulations, andthe lack of reliable templates. Existing approaches typically rely on eitherstrict category-specific priors that restrict generalization, or unconstrainedgenerative models that sacrifice input fidelity. To bridge this gap, wepresent a progressive test-time optimization framework built on 3D Gaus-sian Splatting for high-fidelity 4D animal reconstruction from a singlevideo. Our key insight is that a coarse shape prior suffices when coupledwith a progressive strategy that disentangles articulated pose from non-rigid deformation. Specifically, we employ a symmetry-aware temporalencoding that exploits bilateral cues while absorbing camera estimationdrift and a part-conditioned deformation mechanism guided by learn-able part anchors and a learnable skinning field. Extensive experimentsdemonstrate that our approach generalizes robustly across diverse species,achieving superior geometric accuracy, temporal consistency, and visualfidelity compared to existing baselines, even under severe prior mismatch.Project page: https://syl-322.github.io/ReWild4D/
Show more
E-MOTION: A Dataset for Event-Based Scene Flow Estimation with Independent Moving Objects
Ivan Gutierrez Rodriguez ⋅ Julien Moreau ⋅ Chiara Bartolozzi ⋅ Arren Glover
Scene flow represents the 3D cartesian motion of points inthe world and can be used for applications such as navigation in dy-namic environments, human-robot interaction or non-rigid motion anal-ysis, among others. Despite the potential benefits of event cameras formotion estimation tasks, due to their high temporal resolution and low-latency, not many works have yet addressed their use for scene flowestimation. Progress may be limited by the unconventional data addingcomplexity to established processing pipelines, but also due to the lackof event camera datasets with scene flow ground truth. With the aimof filling this gap, we present E-MOTION: a large and versatile datasetrecorded with high-resolution event cameras suitable for depth, opticalflow and scene flow estimation. E-MOTION features a number of inde-pendent moving objects for which ground truth poses and segmentationmasks are also available. We release a total of 43 sequences with denseground truth maps and poses at 200 Hz. The dataset is available at:https://emotion.hds.utc.fr/
Show more
Monocular Models are Strong Learners for Multi-View Human Mesh Recovery
Haoyu Xie ⋅ Shengkai Xu ⋅ Cheng Guo ⋅ Muhammad Saleem ⋅ Wenhan Wu ⋅ Chen Chen ⋅ Ahmed Helmy ⋅ Pu Wang ⋅ Hongfei Xue
Multi-view human mesh recovery (HMR) is broadly deployed in diverse domains where high accuracy and strong generalization are essential. Existing approaches can be broadly grouped into geometry-based and learning-based methods. However, geometry-based methods (e.g., triangulation) rely on cumbersome camera calibration, while learningbased approaches often generalize poorly to unseen camera con(cid:28)gurations due to the lack of multi-view training data, limiting their performance in real-world scenarios. To enable calibration-free reconstruction that generalizes to arbitrary camera setups, we propose a training-free framework that leverages pretrained single-view HMR models as strong priors, eliminating the need for multi-view training data. Our method (cid:28)rst constructs a robust and consistent multi-view initialization from single-view predictions, and then re(cid:28)nes it via test-time optimization guided by multi-view consistency and anatomical constraints. Extensive experiments demonstrate state-of-the-art performance on standard benchmarks, surpassing multi-view models trained with explicit multiview supervision. • •
Show more
SPARC: Single-Pass Scaling for Motion Forecasting with Conformal Bayesian Last Layers
Sakif Hossain ⋅ Julian Teusch ⋅ Jörg Müller
Human motion forecasters are increasingly accurate and fast, but reliable deployment requires uncertainty estimates that are structured, calibrated, and efficient. Bayesian and ensemble-based uncertainty estimates often require repeated stochastic inference [15,26], while conformal calibration alone does not provide an epistemic signal or preserve trajectory covariance structure [14,50]. We introduce SPARC (SinglePass Adaptive Risk Calibration), a Bayesian–conformal uncertainty layer for motion forecasting. A deterministic MLP backbone predicts the future mean, and a conjugate Bayesian last layer converts time-domain feature leverage into an analytic horizon-wise epistemic scale κ (x). This t scale inflates a graph-temporal Gaussian covariance without changing its correlation structure, and split conformal calibration produces 95% marginal prediction tubes with finite-sample validity under exchangeability. The key interface is the structured factorization κ (x)Σ (x), t str,t which injects feature-space epistemic uncertainty into trajectory densities without Monte Carlo sampling. Across nine dataset/protocol blocks and deterministic, multimodal, and calibration baselines, SPARC ranks first on NLL and on the combined MPJPE+NLL criterion while retaining competitive point accuracy and efficient calibrated tubes. Ranking windows by κ separates high-error cases, making the scale usable as a lightweight risk monitor.
Show more
EgoExoMoCap: Distributed Human Motion Capture via Ego- and Exocentric Body Tracking from Head-Mounted Devices
Jiaxi Jiang ⋅ Bharat Bhatnagar ⋅ Nan Yang ⋅ Lingni Ma ⋅ Sebastian Starke ⋅ Robin Kips ⋅ Nadine Bertsch (Rueegg) ⋅ Christian Holz ⋅ Federica Bogo
Human motion capture from head-mounted devices (HMDs)offers a scalable way to acquire real-world human motion and interac-tion data, which is crucial for applications in embodied AI and VR/AR.Existing approaches focus on either egocentric body tracking, estimat-ing the motion of the subject wearing the device, or exocentric track-ing, capturing the movements of people in the wearer’s surroundings. Sofar, these two paradigms have largely been explored in isolation. In thispaper, we propose a novel distributed framework that jointly leveragesego- and exocentric multi-modal signals for human motion estimationfrom HMDs. Unlike traditional motion capture systems requiring bulkymulti-camera setups or obtrusive mocap suits, our approach, EgoExo-MoCap, is as simple as two (or more) people, each wearing a pair ofsmart glasses. The method leverages head (plus potentially wrist) track-ing signals for accurate estimation of global motion in the 3D world andcombines context-aware image features based on DINOv3 to achieve ro-bustness in the presence of noise and occlusions. Extensive experimentson two in-the-wild datasets show that our approach can robustly recon-struct motion even in challenging scenarios.
Show more
Training-free Controllable Motion Generation under Heterogeneous Constraints
Xiaofei Hui ⋅ Bo Yan ⋅ Haoxuan Qu ⋅ Hossein Rahmani ⋅ Jun Liu
Training-free controllable motion generation has attracted growing interest for enabling flexible constraint enforcement without constraint-specific training. However, existing training-free methods require constraints to be continuous objective-based with differentiable losses, while many real-world requirements are criterion-based and provide only discontinuous, sparse, or even black-box feedback. In this paper, we propose Motion-Inference-as-Control (MIC), the first training-free motion generation framework that handles both continuous objective-based and criterion-based motion constraints under a shared mechanism. The key idea is to cast diffusion-based motion generation as a stochastic control problem. This perspective not only provides principled and practically effective step-wise control laws that support criterion-based constraints without requiring differentiability and naturally accommodate objectivebased constraints as a special case, but also motivates a control-oriented constraint coordination mechanism that adaptively balances and reconciles motion constraints during generation. Experiments across diverse constraint settings demonstrate the effectiveness of our framework.
Show more
Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstruction
Hao Tian ⋅ Chenyangguang Zhang ⋅ Rui Liu ⋅ Wen Shen ⋅ Xiaolin Qin
This paper focuses on a challenging setting of simultaneouslymodeling geometry and appearance of hand-object interaction sceneswithout any object priors. We follow the trend of dynamic 3D GaussianSplatting based methods, and address several significant challenges. Tomodel complex hand-object interaction with mutual occlusion and edgeblur, we present interaction-aware hand-object Gaussians with newlyintroduced optimizable parameters aiming to adopt piecewise linear hy-pothesis for clearer structural representation. Moreover, considering thecomplementarity and tightness of hand shape and object shape duringinteraction dynamics, we incorporate hand information into object de-formation field, constructing interaction-aware dynamic fields to modelflexible motions. To further address difficulties in the optimization pro-cess, we propose a progressive strategy that handles dynamic regions andstatic background step by step. Correspondingly, explicit regularizationsare designed to stabilize the hand-object representations for smooth mo-tion transition, physical interaction reality, and coherent lighting. Exper-iments show that our approach surpasses existing dynamic 3D-GS-basedmethods and achieves state-of-the-art performance in reconstructing dy-namic hand-object interaction.
Show more
OVGGT: O(1) Constant-Cost Streaming Visual Geometry Transformer
Si-yu Lu ⋅ Po-Ting Chen ⋅ Hui-Che Hsu ⋅ Sin-Ye Jhong ⋅ Wen-Huang Cheng ⋅ Yung-Yao Chen
Reconstructing 3D geometry from streaming video requirescontinuous inference under bounded resources. Recent geometric foun-dation models achieve impressive reconstruction quality through all-to-all attention, yet their quadratic cost confines them to short, offline se-quences. Causal-attention variants such as StreamVGGT enable single-pass streaming but accumulate an ever-growing KV cache, exhaustingGPU memory within hundreds of frames and precluding the long-horizondeployment that motivates streaming inference in the first place. Wepresent OVGGT, a training-free framework that bounds both memoryand compute to a fixed budget regardless of sequence length. Our ap-proach combines Self-Selective Caching, which leverages FFN residualmagnitudes to compress the KV cache while remaining fully compatiblewith FlashAttention, with Dynamic Anchor Protection, which shieldscoordinate-critical tokens from eviction to suppress geometric drift overextended trajectories. Extensive experiments on indoor, outdoor, andultra-long datasets show that OVGGT processes arbitrarily long videoswithin a constant VRAM envelope while achieving state-of-the-art 3D ge-ometric accuracy. The code is available at github.com/VAISR/OVGGT.
Show more
OmniCamera: A Unified Framework for Multi-task Video Generation with Arbitrary Camera Control
Yukun Wang ⋅ Ruihuang Li ⋅ Jiale Tao ⋅ Shiyuan Yang ⋅ Liyi Chen ⋅ Zhantao Yang ⋅ Handz Handz ⋅ Yulan Guo ⋅ Shuai Shao ⋅ Qinglin Lu
Video fundamentally intertwines two crucial axes: the dynamic content of a scene and the camera motion through which it is observed. However, existing generation models often entangle these factors, limiting independent control. In this work, we introduce OmniCamera, a unified framework designed to explicitly disentangle and command these two dimensions. This compositional approach enables flexible video generation by allowing arbitrary pairings of camera and content conditions, unlocking unprecedented creative control. To overcome the fundamental challenges of modality conflict and data scarcity inherent in such a system, we present two key innovations. First, we construct OmniCAM, a novel hybrid dataset combining curated real-world videos with synthetic data that provides diverse paired examples for robust multi-task learning. Second, we propose a Dual-level Curriculum Co-Training strategy that mitigates modality interference and synergistically learns from diverse data sources. This strategy operates on two levels: first, it progressively introduces control modalities by difficulties (condition-level), and second, trains for precise control on synthetic data before adapting to real data for photorealism (data-level). As a result, OmniCamera achieves state-of-the-art performance, enabling flexible control for complex camera movements while maintaining superior visual quality.
Show more
VKSR: Scalable Kernel Surface Reconstruction Using Vecchia's Approximation
Maximilian Weiherer ⋅ Chukwudi Williams Umah ⋅ Bernhard Egger
We propose Vecchia Kernel Surface Reconstruction (VKSR),an accurate implicit surface reconstruction method that efficiently scalesrecent kernel-based techniques to large point clouds with millions ofpoints. While existing (global) kernel methods work well in a sparsesetting, due to low-rank approximations, performance degrades quicklywhen presented with dense point clouds sampled from surfaces with highgeometric complexity or large-scale inputs with millions of points. Toovercome this limitation and inspired by the Gaussian Process litera-ture, VKSR uses Vecchia’s approximation instead of low-rank approx-imations, which naturally shifts computation from a global to a locallevel and allows reconstructing 14M+ points in minutes. VKSR achievesstate-of-the-art results on several challenging datasets while retainingkernel methods’ favorable properties when reconstructing sparse inputs.
Show more
RADmesh: Remesh-Aware Mesh Deformation
Nam Anh Dinh ⋅ Itai Lang ⋅ Oded Stein ⋅ Rana Hanocka
We propose a remeshing-enhanced method for generativelydeforming shapes with visual losses. It is intuitive that sufficiently dras-tic deformations of a mesh without changing its triangulation can easilycompromise element quality, even if such large geometry changes maybe semantically desired. Shape deformation methods could thus bene-fit from changing the triangulation; however, this is not done by mostgenerative, text-based, visually-supervised mesh deformation methods.Remeshing is a discrete operation, proven to be especially challengingto couple with the notoriously noisy supervision signal provided by vi-sual losses. We propose a vertex-based deformation optimization quantitycapable of large deformations and robustness to such noise; we period-ically remesh using an isotropic remesher that interpolates and carriesforward the deformation optimization state. This enables continuous,geometry-informed progress in coarse-to-fine addition of resolution. Theresulting shapes’ triangulations fit their optimized geometry and haveneat isotropic elements. Further, our method is localizable, able to grownew features on a base shape with expressive detail, leaving the rest un-changed. We showcase the effectiveness of our method on a variety ofshapes and prompts, both local and global deformations, and demon-strate its superior visual quality and triangle efficiency. Our project pageis at https://threedle.github.io/radmesh.
Show more
CameraAnything: Refilming Videos with Arbitrary Camera Control
Yixuan Li ⋅ Yanhong Zeng ⋅ Ka Leong Cheng ⋅ Jiayi Zhu ⋅ Hanlin Wang ⋅ Wen Wang ⋅ Yihao Meng ⋅ Hao Ouyang ⋅ Qiuyu Wang ⋅ Yue Yu ⋅ Zidong Wang ⋅ Yiyuan Zhang ⋅ Yujun Shen ⋅ Dahua Lin
We introduce CameraAnything, the first unified frame-work for camera controlled video editing that enables joint control ofboth intrinsic and extrinsic camera parameters. Existing approaches ei-ther rely on expensive 3D reconstruction to achieve full camera function-ality or restrict editing to extrinsic parameter manipulation. Moreover,the coupled influence of intrinsic and extrinsic parameters on video ap-pearance makes disentangled modeling particularly challenging. To ad-dress this, we adopt per-pixel Plücker ray injection alongside resolution-aware 3D RoPE in self-attention, building both camera conditioning andspatial positional encoding on the target latent to jointly control cam-era position, focal length, and native resolution editing without croppingor outpainting. To overcome the scarcity of paired training data, wefurther develop a scalable synthetic pipeline that constructs diverse dy-namic scenes through structured multi-camera recording and generatessynchronized videos with varied camera configurations. With a tailoredorthogonal training strategy, CameraAnything enables expressive videoreshooting with arbitrary viewpoint control, focal length adjustment, res-olution adaptation, and multi-shot transitions within a single generationprocess, offering strong practical value for cinematic video editing andcross-platform content adaptation in video production.
Show more
Face Anything: 4D Face Reconstruction from Any Image Sequence
Umut Kocasarı ⋅ Simon Giebenhain ⋅ Richard Shaw ⋅ Matthias Niessner
Accurate reconstruction and tracking of dynamic human facesfrom image sequences is challenging because non-rigid deformations, ex-pression changes, and viewpoint variations occur simultaneously, creat-ing significant ambiguity in geometry and correspondence estimation. Wepresent a unified method for high-fidelity 4D facial reconstruction basedon canonical facial point prediction, a representation that assigns eachpixel a normalized facial coordinate in a shared canonical space. Thisformulation transforms dense tracking and dynamic reconstruction intoa canonical reconstruction problem, enabling temporally consistent ge-ometry and reliable correspondences within a single feed-forward model.By jointly predicting depth and canonical coordinates, our method en-ables accurate depth estimation, temporally stable reconstruction, dense3D geometry, and robust facial point tracking within a single architec-ture. We implement this formulation using a transformer-based modelthat jointly predicts depth and canonical facial coordinates, trained us-ing multi-view geometry data that non-rigidly warps into the canonicalspace. Extensive experiments on image and video benchmarks demon-strate state-of-the-art performance across reconstruction and trackingtasks, achieving approximately 3× lower correspondence error and fasterinference than prior dynamic reconstruction methods, while improvingdepth accuracy by 16%. These results highlight canonical facial point pre-diction as an effective foundation for unified feed-forward 4D facial recon-struction. Website: https://kocasariumut.github.io/FaceAnything/
Show more
RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Video
Ulrich Prestel ⋅ Stefan Andreas Baumann ⋅ Nick Stracke ⋅ Bjorn Ommer
Self-supervised novel view synthesis (NVS) remains challenging to scale, despite the abundance of video data, largely due to the brittleness of training on realistic videos and the hard-to-predict scaling behavior of multi-network system designs. We introduce RayDer, a unified, feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone, turning self-supervised NVS into a well-posed single-model scaling problem. A minimal dynamic state, treated as a nuisance factor, absorbs time-varying content and enables stable training on unconstrained real-world video. Importantly, RayDer keeps static-scene NVS as its target task: dynamic content is leveraged purely as scalable supervision, not reconstructed as in dynamicscene (4D) NVS. Across multiple model sizes and orders of magnitude in data, RayDer exhibits clean power-law scaling with data and compute, and outperforms static-scene data mixtures. On a large number of benchmarks, RayDer achieves strong zero-shot open-set performance competitive with state-of-the-art supervised approaches. Project Page: https://compvis.github.io/rayder.
Show more
HoloTetSphere: Unified TetSphere Mesh Reconstruction for Physical Simulations
Yaqiao Dai ⋅ Renjiao Yi ⋅ Zhirui Gao ⋅ Wei Chen ⋅ Kai Xu ⋅ Chenyang Zhu
Standard pipelines for physics-ready 3D reconstruction relyon a decoupled two-stage paradigm: extracting surface geometry followedby an error-prone tetrahedralization process. While recent Lagrangianmethods like TetSphere Splatting attempt to bypass this by directlyoptimizing volumetric primitives, their homeomorphic constraints pre-vent topology-adaptive optimization. Consequently, they produce dis-joint tetrahedra rather than a single connected mesh, rendering thestructures unsuitable for further physical simulations. To address this,we propose a topology-adaptive framework for holistic tetrahedral meshreconstruction through end-to-end topological and geometric optimiza-tion. First, by coupling Gaussian spheres to tetrahedral elements andleveraging edge connections, we estimate a continuous opacity field fordifferentiable element pruning. Next, jointly minimizing mesh smoothingenergy and multi-view Gaussian rendering error drives alternating geo-metric refinement while preserving topological adaptivity. Consequently,our approach effectively constructs a unified and topologically coherenttetrahedral mesh. Extensive experiments demonstrate that our methodoutperforms state-of-the-art techniques by achieving superior geometricaccuracy and producing coherent, single-connected tetrahedral meshes,thereby effectively bypassing the error-prone conventional tetrahedraliza-tion step for reconstructed surface meshes and streamlining downstreamphysical simulation.
Show more
A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models
Nuo Chen ⋅ Lulin Liu ⋅ Zihao Li ⋅ Ziyao Zeng ⋅ Zihao Zhu ⋅ Wenyan Cong ⋅ Junyuan Hong ⋅ Yunhao Yang ⋅ Zhengzhong Tu ⋅ Yan Wang ⋅ Boris Ivanovic ⋅ Marco Pavone ⋅ Zhangyang Wang ⋅ Yang Zhou ⋅ Zhiwen Fan
Generative world models hold immense promise as scalablesimulators for autonomous systems, particularly for synthesizing rare butsafety-critical multi-agent interactions, such as vehicle collisions. How-ever, current evaluation paradigms index heavily on visual fidelity and se-mantic alignment, leaving a critical blind spot: they cannot reliably quan-tify whether generated dynamics actually obey the fundamental physicallaws required for reliable simulation. Assessing this physical plausibilityis inherently difficult due to a lack of physical metrics and the challengeof extracting metric-scale kinematics from uncalibrated video rollouts.To bridge this gap, we introduce CrashTwin, a physics-grounded eval-uation framework designed to stress-test the physical trustworthiness ofworld models. CrashTwin couples a diverse dataset of multi-agent colli-sion scenarios, comprising 25K controllable synthetic and 12K in-the-wildreal-world collision sequences with a novel calibration-free reconstruc-tion pipeline, enabling the recovery of 3D physical attributes directlyfrom world model rollouts. We propose a diagnostic suite that systemat-ically evaluates three dimensions: spatio-temporal consistency, momen-tum and kinetic energy conservation, and world-dynamics integrity. Ex-tensive benchmarking of state-of-the-art models reveals a crucial insight:high perceptual quality frequently masks severe physical violations dur-ing complex interactions. By quantitatively exposing these failure modes,CrashTwin provides a vital diagnostic tool for developing physicallygrounded world models capable of reliable real-world simulation. Codeand dataset are available at: https://github.com/phai-lab/CrashTwin.
Show more
Grounding World Simulation Models in a Real-World Metropolis
Junyoung Seo ⋅ Hyunwook Choi ⋅ Minkyung Kwon ⋅ Jinhyeok Choi ⋅ Siyoon Jin ⋅ Gayoung Lee ⋅ Junho Kim ⋅ JoungBin Lee ⋅ Geonmo Gu ⋅ Dongyoon Han ⋅ Sangdoo Yun ⋅ Seungryong Kim ⋅ Jin-Hwa Kim
What if a world simulation model could render not an imag-ined environment but a city that actually exists? Prior generative worldmodels synthesize visually plausible yet artificial environments by imag-ining all content. We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul. SWM anchors au-toregressive video generation through retrieval-augmented conditioningon nearby street-view images. However, this design introduces severalchallenges, including temporal misalignment between retrieved referencesand the dynamic target scene, limited trajectory diversity and data spar-sity from vehicle-mounted captures at sparse intervals. We address thesechallenges through cross-temporal pairing, a large-scale synthetic datasetenabling diverse camera trajectories, and a view interpolation pipelinethat synthesizes coherent training videos from sparse street-view images.We further introduce a Virtual Lookahead Sink to stabilize long-horizongeneration by continuously re-grounding each chunk to a retrieved im-age at a future location. We evaluate SWM against recent video worldmodels across three cities: Seoul, Busan, and Ann Arbor. SWM out-performs existing methods in generating spatially faithful, temporallyconsistent, long-horizon videos grounded in actual urban environmentsover trajectories reaching hundreds of meters, while supporting diversecamera movements and text-prompted scenario variations.
Show more
InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360° Image
Gwanhyeong Koo ⋅ Hyunsu Kim ⋅ Youngji Kim ⋅ Taejae Lee ⋅ Siwoo Lim ⋅ Sunjae Yoon ⋅ Suyong Yeon ⋅ Chang D. Yoo
Recent advances in single image-to-3D generation have en-abled high-quality asset synthesis, yet extending these capabilities toindoor scene generation remains challenging. Existing methods focus onasset-level generation while neglecting the structural layout, which isessential for downstream applications and serves as the spatial anchorfor grounding assets. However, a single image with a limited field ofview lacks the spatial coverage to recover a coherent global layout. Tothis end, we use a 360° image represented in equirectangular projec-tion (ERP) and propose InSpace, a structure-aware framework for 3Dindoor scene generation. InSpace comprises three stages: (1) estimat-ing partial scene geometry as spatial priors, (2) generating coarse scene* †Work done during an internship at NAVER LABS. Co-corresponding authorsArtifactsMisplacementMisplacementERP Image ERP ImageInput Result Floating Input Result ResultResultSceneGen SAM3D InSpace (Ours)(a) Current Single Image to 3D Scene Generation (b) ERP Image to 3D Scene GenerationFig. 2: (a) Existing single-image methods generate individual assets without structurallayout, causing floating, misplacement, and artifacts. (b) InSpace uses an ERP imageto generate complete indoor scenes with structural layout and well-grounded assets.structure with view-selective cross-attention, and (3) producing detailedlayout and asset geometry with textures through a global-local hybridattention, using flow matching. We also propose ERP-FRONT, a pairedERP-Image-to-3D indoor scene dataset based on 3D-FRONT. Exper-iments show that InSpace generates complete 3D indoor scenes withstructural layout, along with separate textured assets from a single ERPimage, achieving strong performance across 3D and 2D metrics.
Show more
UniDDT: Unifying Multimodal Understanding and Generation with Decoupled Diffusion Transformer
Shuai Wang ⋅ Liang Li ⋅ Yang Chen ⋅ Ruopeng Gao ⋅ Yao Teng ⋅ Limin Wang
Unified Multimodal Models (UMMs) have emerged as a critical direction for general-purpose multimodal intelligence, integrating understanding and generation into a single framework. However, existing UMMs face prominent challenges: (1) the inherent learning conflicts between visual understanding and generation tasks, leading to suboptimal modeling in both tasks; (2) different understanding and generation visual spaces impeding scalability; (3) over-reliance on task-specific data that neglects the duality of text-image understanding and generation. To address these challenges, we propose UniDDT, which leverages a Noisy ViT encoder along with a LLM to unify semantic encoding for visual generation and understanding tasks, while employing a separate diffusion decoder to decouple diffusion decoding from text decoding. With this Noisy ViT encoder, UniDDT is able to leverage the latent space as a unified visual representation, enabling seamless compatibility between understanding and generation tasks. Thus, the scalability within the generation tasks and the semantic expressiveness within understanding tasks can be balanced. Also, we construct dual data structures from the same image-text pairs, fostering interdependence between the generation and understanding data to exploit their inherent duality. Extensive experiments demonstrate that UniDDT achieves effective unification of multimodal understanding and generation with enhanced semantic consistency and scalability. For visual generation tasks, our UniDDT achieves 0.86 GenEval score and 86.9 DPG overall score. For multimodal understanding tasks, our UniDDT achieves 1699.5 score on MME benchmark and 76.5 overall score on SEEDbench.
Show more
Silhouette-based Gait Foundation Model
Dingqiang Ye ⋅ Chao Fan ⋅ Kartik Narayan ⋅ Bingzhe Wu ⋅ Chengwen Luo ⋅ Jianqiang Li ⋅ Vishal Patel
Gait patterns play a critical role in human identification andhealthcare analytics, yet current progress remains constrained by small,narrowly designed models that fail to scale or generalize. Building a uni-fied gait foundation model requires addressing two longstanding barriers:(a) Scalability – Why have gait models historically failed to follow em-pirical scaling trends? (b) Generalization – Can one model serve thediverse gait tasks that have traditionally been studied in isolation? Weintroduce FoundationGait, the first scalable, self-supervised pretrain-ing framework for vision-based gait understanding. Its largest versionhas nearly 0.13 billion parameters and is pretrained on 12 public gaitdatasets comprising over 2 million walking sequences. Extensive exper-iments demonstrate that FoundationGait, with or without fine-tuning,performs robustly across a wide spectrum of gait datasets, conditions,tasks (e.g., human identification, scoliosis screening, depression predic-tion, and attribute estimation), and even input modality. Notably, itachieves 48.0% self-supervised rank-1 accuracy on the challenging in-the-wild Gait3D dataset (1,000 test subjects) and 64.5% on the largestin-the-lab OU-MVLP dataset (5,000+ test subjects), setting a new mile-stone in robust gait recognition. These results establish FoundationGaitas a strong and versatile foundation for future gait research. All codeand models: https://github.com/ShiqiYu/OpenGait.
Show more
CMCC-ReID: Cross-Modality Clothing-Change Person Re-Identification
Xu Haoxuan ⋅ Hanzi Wang ⋅ Guanglin Niu
Person Re-Identification (ReID) faces severe challenges frommodality discrepancy and clothing variation in long-term surveillancescenario. While existing studies have made significant progress in ei-ther Visible-Infrared ReID (VI-ReID) or Clothing-Change ReID (CC-ReID), real-world surveillance system often face both challenges simul-taneously. To address this overlooked yet realistic problem, we definea new task, termed Cross-Modality Clothing-Change Re-Identification(CMCC-ReID), which targets pedestrian matching across variations inboth modality and clothing. To advance research in this direction, weconstruct a new benchmark SYSU-CMCC, where each identity is cap-tured in both visible and infrared domains with distinct outfits, reflectingthe dual heterogeneity of long-term surveillance. To tackle CMCC-ReID,we propose a Progressive Identity Alignment Network (PIA) that pro-gressively mitigates the issues of clothing variation and modality dis-crepancy. Specifically, a Dual-Branch Disentangling Learning (DBDL)module separates identity-related cues from clothing-related factors toachieve clothing-agnostic representation, and a Bi-Directional PrototypeLearning (BPL) module performs intra-modality and inter-modality con-trast in the embedding space to bridge the modality gap while furthersuppressing clothing interference. Extensive experiments on the SYSU-CMCC dataset demonstrate that PIA establishes a strong baseline forthis new task and significantly outperforms existing methods.
Show more
RealGen: Photorealistic Text-to-Image Generation via Detector-Guided Rewards
ye junyan ⋅ Leqi Zhu ⋅ Yuncheng Guo ⋅ Dongzhi Jiang ⋅ Zilong Huang ⋅ Yifan Zhang ⋅ Zhiyuan Yan ⋅ Haohuan Fu ⋅ Conghui He ⋅ Weijia Li
Rapid advancements in image generation have led models likeGPT-Image and Qwen-Image to excel in text-to-image consistency andworld knowledge. However, they still struggle with photorealism, oftenproducing "fake" images laden with obvious AI artifacts, such as overlysmooth skin or unnatural oily sheens. To recapture the original goal of"indistinguishable-from-reality" generation, we propose RealGen, a photo-realistic text-to-image framework. RealGen integrates an LLM componentfor prompt optimization and a diffusion model for realistic image genera-tion. Inspired by adversarial generation, RealGen introduces a "DetectorReward" mechanism, which quantifies artifacts and assesses realism usingboth semantic and feature-level synthetic image detectors. We leveragethis reward with the GRPO algorithm to optimize the entire generationpipeline, significantly enhancing image realism and detail. Furthermore,we propose RealBench, an automated evaluation benchmark employingDetector-Scoring and Arena-Scoring. It enables human-free photorealismassessment, yielding results that are more accurate and aligned with realuser experience. Experiments show that RealGen produces photorealisticimages with enhanced realism, detail, and aesthetics.
Show more
QVAM: Query-guided View-aware Adaptive Modulation for Aerial-Ground Person Re-Identification
Mengfei Zhou ⋅ Lin Wan
Aerial–Ground Person Re-Identification (AGPReID) matches identities across unmanned aerial vehicles (UAVs) and ground cameras but suffers from extreme viewpoint gaps. Most existing methods rely on coarse binary aerial/ground labels and rigid orthogonality to disentangle view and identity features. We identify an important limitation of prior methods: the oversimplified binary-label design cannot fully capture continuous aerial viewpoint shifts. Moreover, rigid orthogonality constraints may further suppress identity cues. To address these issues, we propose Query-guided View-aware Adaptive Modulation (QVAM) for identity-preserving cross-view matching. Specifically, a View-aware Decoder (VAD) distills fine-grained viewpoint cues from patch tokens using learnable view queries. Guided by these cues, Adaptive Feature Modulation (AFM) predicts query-conditioned masks to suppress view-biased responses while preserving identity discrimination. A Cross-View Prototype Alignment (CVPA) loss further aligns modulated features at batch and memory levels with dual-view memory banks. Extensive experiments on AG-ReID, AG-ReIDv2, and CARGO show that QVAM achieves stateof-the-art performance, improving the previous best results by +10.26% Rank-1/+10.83% mAP on CARGO-ALL and +2.62% Rank-1/+2.37% mAP on AG-ReID A→G. The code is available at https://github.com/ Sakuraandroxy/QVAM.
Show more
Scaling Multi-Reference Image Generation with Dynamic Reward Optimization
Wenwang Huang ⋅ Yusen Fu ⋅ Mengfei Huang ⋅ Junjie Wang ⋅ Yulin Li ⋅ Gan Liu ⋅ Jing Cai ⋅ Yancheng He ⋅ Zhuotao Tian
While personalized image generation has achieved remark-able progress, multi-reference image generation (MRIG) remains a chal-lenging task. Most existing benchmarks fail to adequately evaluate com-plex MRIG scenarios, hindering further progress in this area. To bet-ter assess model performance on complex MRIG tasks, we introduceOmniRef-Bench, a benchmark that covers complex combinations of ref-erence image types and a large number of reference images. Evaluationson OmniRef-Bench show that mainstream open-source models strugglein complex MRIG scenarios, and their performance deteriorates signifi-cantly as the number of mixed-type reference images increases. To ad-dress this issue, we propose DyRef, a two-stage training framework.In the first stage, supervised fine-tuning equips the model with the ba-sic capability to handle complex MRIG tasks. In the second stage, weintroduce Difficulty-aware Advantage Reweighting (DAR) and Discrim-inative Reward Scaling (DRS). DAR dynamically adjusts the optimiza-tion objective to improve performance when handling a large numberof mixed-type reference images. DRS enlarges intra-group reward differ-ences for more effective policy optimization. Experiments demonstratethat DyRef significantly improves the performance of open-source modelson OmniRef-Bench and single-image editing benchmarks, demonstratingthe effectiveness and generalization capability of our approach. Our codeis available at https://github.com/Weistrass/DyRef.
Show more
Divide and Align: Disentangled Vision-Language Learning for Text-Based Person Anomaly Search
Alex Ergasti ⋅ Tomaso Fontanini ⋅ Claudio Ferrari ⋅ Massimo Bertozzi ⋅ Andrea Prati
Text-based person anomaly search (TPAS) refers to the taskof retrieving people exhibiting normal or anomalous behaviors from nat-ural language descriptions. Existing TPAS models often learn a singlejoint embedding where appearance, action, and background informationare entangled, causing over-reliance on identity cues, poor alignment foraction-centric queries, and limited semantic connection between actionsand places where they occur. To address these issues, we propose Se-mantic Decoupled Alignment (SeDA), a disentangled vision–languageretrieval framework that explicitly factorizes both visual and textualrepresentations into appearance, action, and background components.SeDA introduces Semantic Token Projection, which derives three se-mantic queries from the global [CLS] token, softly aggregates modal-ity tokens relevant to each factor, and recomposes the resulting factortokens into a compact retrieval embedding. To enforce factor-specific se-mantics, we decompose each caption into appearance/action/backgroundsub-captions and supervise the corresponding tokens with a Feature De-coupling Loss, combined with contrastive and image–text matching ob-jectives. On the Person Anomaly Benchmark (1M pairs), SeDA achieves86.45% R@1 (+1.52 over SOTA), improves average multi-weather R@1by +2.43, and gains +3.74 R@1 under out-of-distribution evaluation.https://github.com/ErgastiAlex/SeDA
Show more
AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillation
Yuchao Gu ⋅ Guian Fang ⋅ Yuxin Jiang ⋅ Weijia Mao ⋅ Song Han ⋅ Han Cai ⋅ Mike Zheng Shou
Few-step video generation has been significantly advancedby consistency distillation. However, the performance of consistency-distilled models often degrades as more sampling steps are allocatedat test time, limiting their effectiveness for any-step video diffusion.We argue that this limitation arises because consistency distillation re-places the original probability-flow ODE trajectory with a consistency-sampling trajectory, weakening the desirable test-time scaling behaviorof ODE sampling. To address this limitation, we introduce AnyFlow,the first any-step video diffusion distillation framework based on flowmaps. Instead of distilling a model for only a few fixed sampling steps,AnyFlow optimizes the full ODE sampling trajectory. To this end, weshift the distillation target from endpoint consistency mapping (zt → z0 )to flow-map transition learning (zt → zr ) over arbitrary time inter-vals. We further propose Flow Map Backward Simulation, whichdecomposes a full Euler rollout into shortcut flow-map transitions, en-abling efficient on-policy distillation that reduces test-time errors (i.e.,discretization error in few-step sampling and exposure bias in causalgeneration). Extensive experiments across both bidirectional and causalarchitectures, at scales ranging from 1.3B to 14B parameters, demon-strate that AnyFlow achieves performance comparable to or better thanconsistency-based counterparts in the few-step regime, while support-ing flexible and scalable sampling under varying step budgets. Code isreleased at https://github.com/NVLabs/AnyFlow.
Show more
PADFormer: Pose-agnostic Anomaly Detection from Sparse View Images
Ruiqi Wang ⋅ Yiming Qian ⋅ FENGGEN YU ⋅ Yuxuan Lu ⋅ Dakuo Wang ⋅ Hao Richard Zhang ⋅ Jing Huang
Pose-agnostic Anomaly Detection (PAD) remains challeng-ing as anomalies can appear under arbitrary viewpoints, requiring meth-ods to handle significant pose variations. Existing approaches rely oncomplex 3D reconstruction, which are computationally expensive and re-quire extensive multi-view data. We propose PADFormer, a novel image-space approach that leverages Vision Transformer (ViT) to directly re-construct anomaly-free versions of query images while preserving poseinformation. Our key insight is to adapt cross-view masked reconstruc-tion for anomaly detection through training exclusively on normal data,combined with dynamic patch selection and spatial alignment mecha-nisms that enable effective learning from sparse reference views undersignificant pose variations. During inference, we perform multiple for-ward passes with different masking patterns to generate an ensembleof anomaly-free reconstructions, ensuring comprehensive coverage of thequery image. Anomalies are detected by comparing these reconstructionswith the query image. PADFormer achieves state-of-the-art results on thePAD benchmark while maintaining comparable performance on classicfew-shot anomaly detection (FSAD) tasks, demonstrating superior effi-ciency and generalization without requiring 3D reconstruction.
Show more
Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusion
Hau-Shiang Shiu ⋅ Chin-Yang Lin ⋅ Zhixiang Wang ⋅ Chi-Wei Hsiao ⋅ Po-Fan Yu ⋅ Yu-Chih Chen ⋅ Yu-Lun Liu
Diffusion-based video super-resolution (VSR) methods deliver strong perceptual quality but are often unsuitable for latency-sensitive scenarios due to reliance on future frames and expensive multi-step denoising. We propose Stream-DiffVSR, a causally conditioned diffusion framework for efficient online VSR. Operating strictly on past frames, Stream-DiffVSR integrates a four-step distilled denoiser for fast inference, an Auto-regressive Temporal Guidance (ARTG) module that injects motion-aligned cues during latent denoising, and a lightweight temporalaware decoder with a Temporal Processor Module (TPM) to enhance detail and temporal coherence. Unlike chunk-wise streaming inference, our strictly frame-by-frame causal design avoids sequence-level waiting, substantially reducing time-to-first-frame and end-to-end latency. Stream-DiffVSR processes 720p frames in 0.328 seconds on an RTX 4090 and consistently outperforms prior diffusion-based baselines. Compared with the online state-of-the-art TMP, it improves perceptual quality (LPIPS +0.095). Compared with prior diffusion-based VSR methods such as MGLD-VSR [104], it reduces per-frame runtime by over 130×. Moreover, Stream-DiffVSR substantially lowers time-to-firstframe for diffusion-based VSR, reducing initial delay from over 4600 seconds to 0.328 seconds, making diffusion-based VSR markedly more practical for low-latency online and streaming deployment. Project page: https://jamichss.github.io/stream-diffvsr-project-page/
Show more
RiO-DETR: DETR for Real-time Oriented Object Detection
Zhangchi Hu ⋅ Yifan Zhao ⋅ Yansong Peng ⋅ Wenzhang SUN ⋅ Xiangchen Yin ⋅ Jie Chen ⋅ Peixi Wu ⋅ Hebei Li ⋅ xinghao wang ⋅ Dongsheng Jiang ⋅ Xiaoyan Sun
We present RiO-DETR: DETR for Real-time OrientedObject Detection, the first real-time oriented detection transformer tothe best of our knowledge. Adapting DETR to oriented bounding boxes(OBBs) poses three challenges: semantics-dependent orientation, angleperiodicity that breaks standard Euclidean refinement, and an enlargedsearch space that slows convergence. RiO-DETR resolves these issueswith task-native designs while preserving real-time efficiency. First, wepropose Content-Driven Angle Estimation by decoupling angle from po-sitional queries, together with Rotation-Rectified Orthogonal Attentionto capture complementary cues for reliable orientation. Second, Decou-pled Periodic Refinement combines bounded coarse-to-fine updates witha Shortest-Path Periodic Loss for stable learning across angular seams.Third, Oriented Dense O2O injects angular diversity into dense super-vision to speed up angle convergence at no extra cost. Extensive ex-periments on DOTA-1.0, DIOR-R, and FAIR-1M-2.0 demonstrate RiO-DETR establishes a new speed–accuracy trade-off for real-time orienteddetection. Code is available at https://github.com/RicePasteM/RiO-DETR.
Show more
EditHF-1M: A Million-Scale Rich Human Preference Feedback for Image Editing
Zitong Xu ⋅ Huiyu Duan ⋅ Zhongpeng Ji ⋅ Xinyun Zhang ⋅ Yutao Liu ⋅ Xiongkuo Min ⋅ Ke Gu ⋅ Jian Zhang ⋅ Shusong Xu ⋅ Jinwei Chen ⋅ Bo Li ⋅ Guangtao Zhai
Recent text-guided image editing (TIE) models have achievedremarkable progress, while many edited images still suffer from issuessuch as artifacts, unexpected editings, unaesthetic contents. Althoughsome benchmarks and methods have been proposed for evaluating editedimages, scalable evaluation models are still lacking, which limits the de-velopment of human feedback reward models for image editing. To ad-dress the challenges, we first introduce EditHF-1M, a million-scale im-age editing dataset with over 29M human preference pairs and 148Khuman mean opinion ratings, both evaluated from three dimensions,i.e., visual quality, instruction alignment, and attribute preservation.Based on EditHF-1M, we propose EditHF, a multimodal large lan-guage model (MLLM) based evaluation model, to provide human-alignedfeedback from image editing. Finally, we introduce EditHF-Reward,which utilizes EditHF as a reward signal to optimize the text-guidedimage editing models through reinforcement learning. Extensive experi-ments show that EditHF achieves superior alignment with human prefer-ences and demonstrates strong generalization on other datasets. Further-more, we fine-tune the Qwen-Image-Edit using EditHF-Reward, achiev-ing significant performance improvements, which demonstrates the abil-ity of EditHF to serve as a reward model to scale-up the image editing.Both the dataset and code will be released in our GitHub repository:https://github.com/IntMeGroup/EditHF.
Show more
WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing
Hui Zhang ⋅ Juntao Liu ⋅ Zongkai Liu ⋅ liqiang niu ⋅ Fandong Meng ⋅ Zuxuan Wu ⋅ Yu-Gang Jiang
Instruction-based image editing aims to modify specific con-tent within existing images according to user-provided instructions whilepreserving non-target regions. Beyond traditional object- and style-centricmanipulation, text-centric image editing focuses on modifying, translating,or rearranging textual elements embedded within images. However, exist-ing leading models often struggle to execute complex text editing precisely,frequently producing blurry or hallucinated characters. We attribute thesefailures primarily to the lack of specialized training paradigms tailoredfor text-centric editing, as well as the absence of large-scale datasetsand standardized benchmarks necessary for a closed-loop training andevaluation system. To address these limitations, we present WeEdit, a sys-tematic solution encompassing a scalable data construction pipeline, twobenchmarks, and a tailored two-stage training strategy. Specifically, wepropose a novel HTML-based automatic editing pipeline, which generates330K training pairs covering diverse editing operations and 15 languages,accompanied by standardized bilingual and multilingual benchmarks forcomprehensive evaluation. On the algorithmic side, we employ glyph-guided supervised fine-tuning to inject explicit spatial and content priors,followed by a multi-objective reinforcement learning stage to align genera-tion with instruction adherence, text clarity, and background preservation.Extensive experiments demonstrate that WeEdit outperforms previousopen-source models by a clear margin across diverse editing operations.
Show more
Align and Segment: Unsupervised Learning for Building Segmentation From Misaligned Labels
Venkanna Babu Guthula ⋅ Oswin Krause ⋅ Dimitri Gominski ⋅ Hui Zhang ⋅ Johan Mottelson ⋅ Ankit Kariryaa ⋅ Nico Lang ⋅ Christian Igel
Supervised learning for image segmentation typically requires spatially aligned image and label sets. When images and labels originate from different sources, the pairing may be misaligned, which can significantly deteriorate the performance of the learned models. This is especially common in remote sensing, when aerial or satellite images are co-registered with labels from another source (e.g., OpenStreetMap). In this work, we propose a novel approach for training on misaligned labels, where we simultaneously learn the label alignment. Our align and segment (AnS) approach builds on the spatial transformer module to transform the misaligned labels using an affine transformation to provide a better learning target for a canonical semantic segmentation network. We prevent shortcut learning of misaligned labels in these semantic segmentation networks through a self-supervised regularization loss and show that it is complementary to data augmentation, especially for systematically misaligned training data. A decisive characteristic of our AnS approach is that it learns without requiring any “golden” labels. We experimentally show on both synthetic and real-world data from different cities that our approach enables high-quality building segmentation and precise label-image alignment at the same time. Code and derived datasets are available at https://github.com/venkanna37/align-and-segment.
Show more
Dress-ED: Instruction-Guided Editing for Virtual Try-On and Try-Off
Fulvio Sanguigni ⋅ Davide Lobba ⋅ Bin Ren ⋅ Marcella Cornia ⋅ Nicu Sebe ⋅ Rita Cucchiara
Recent advances in Virtual Try-On (VTON) and VirtualTry-Off (VTOFF) have greatly improved photo-realistic fashion synthesisand garment reconstruction. However, existing datasets remain static,lacking instruction-driven editing for controllable and interactive fash-ion generation. In this work, we introduce the Dress Editing Dataset(Dress-ED), the first large-scale benchmark that unifies VTON, VTOFF,and text-guided garment editing within a single framework. Each sam-ple in Dress-ED includes an in-shop garment image, the correspondingperson image wearing the garment, their edited counterparts, and anatural-language instruction of the desired modification. Built througha fully automated multimodal pipeline that integrates MLLM-basedgarment understanding, diffusion-based editing, and LLM-guided ver-ification, Dress-ED comprises over 146k verified quadruplets spanningthree garment categories and seven edit types, including both appear-ance (e.g., color, pattern, material) and structural (e.g., sleeve length,neckline) modifications. Based on this benchmark, we further propose aunified multimodal diffusion framework that jointly reasons over linguisticinstructions and visual garment cues, serving as a strong baseline forinstruction-driven VTON and VTOFF.
Show more
Exclusivity-Guided Mask Learning for Semi-Supervised Crowd Instance Segmentation and Counting
Jiyang Huang ⋅ Hongru Chen ⋅ Wei Lin ⋅ Jia Wan ⋅ Antoni Chan
Semi-supervised crowd analysis is a prominent area of re-search, as unlabeled data are typically abundant and inexpensive toobtain. However, traditional point-based annotations constrain perfor-mance because individual regions are inherently ambiguous, and con-sequently, learning fine-grained structural semantics from sparse anno-tations remains an unresolved challenge. In this paper, we first pro-pose an Exclusion-Constrained Dual-Prompt SAM (EDP-SAM), basedon our Nearest Neighbor Exclusion Circle (NNEC) constraint, to gener-ate mask supervision for current datasets. With the aim of segmentingindividuals in dense scenes, we then propose Exclusivity-Guided MaskLearning (XMask), which enforces spatial separation through a discrim-inative mask objective. Gaussian smoothing and a differentiable centersampling strategy are utilized to improve feature continuity and train-ing stability. Building on XMask, we present a semi-supervised crowdcounting framework that uses instance mask priors as pseudo-labels,which contain richer shape information than traditional point cues. Ex-tensive experiments on the ShanghaiTech A, UCF-QNRF, and JHU++datasets (using 5%, 10%, and 40% labeled data) verify that our end-to-end model achieves state-of-the-art semi-supervised segmentation andcounting performance, effectively bridging the gap between counting andinstance segmentation within a unified framework. Code can be found athttps://github.com/JoyceeH0127/ECCV2026XMask.
Show more
Instance Segmentation as Tracking: A New Paradigm for Multi-Small-Object Tracking with Event Cameras
Nuo Chen ⋅ Shiman He ⋅ Boyang Li ⋅ Yingqian Wang ⋅ Chao Xiao ⋅ QianYin QianYin ⋅ Ruojing Li ⋅ Yihang Luo ⋅ Wei An ⋅ Miao Li
Multiple small object tracking (MSOT) is critical for ap-plications such as anti-UAV systems and security surveillance, yet tra-ditional frame-based cameras struggle to track fast-moving small ob-jects in complex environments due to their low frame rates and lim-ited dynamic range. Event cameras, with their ability to continuouslyrecord subtle brightness changes, can naturally overcome these limita-tions. However, most existing event-based tracking methods follow a“convert-then-detect-and-track” pipeline. This pipeline sacrifices the in-herent high temporal resolution of event data, leading to fragmentedtrajectories of fast-moving objects. Moreover, it introduces significantbackground redundancy during framing, which reduces computational ef-ficiency. To handle these issues, we introduce “instance segmentationas tracking”, a novel paradigm that formulates event-based MSOT asan instance segmentation task in the spatio-temporal dimension. Follow-ing this paradigm, we first design Ev-ISNet, which leverages 3D sparseconvolutions to extract per-voxel features while simultaneously predict-ing object confidence and motion direction. Then we construct an eventgraph and progressively cluster trajectory instances by predicting edge-wise instance affinities. By leveraging intra-graph and inter-graph associ-ation modules, our method achieves highly efficient streaming inference.To address the lack of large-scale benchmarks for event-based MSOT,we build EV-UAV-Track, a comprehensive dataset featuring per-eventinstance-level annotations. Extensive experiments demonstrate that Ev-ISNet consistently outperforms state-of-the-art MSOT methods, achiev-ing over 30% improvement in MOTA score compared to frame-basedtracking methods.
Show more
GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing
Zifeng Zhu ⋅ Jiaming Han ⋅ Jiaxiang Zhao ⋅ Minnan Luo ⋅ Xiangyu Yue
While Diffusion Large Language Models (DLLMs) have demon-strated remarkable capabilities in multi-modal generation, performingprecise, training-free image editing remains an open challenge. Unlikecontinuous diffusion models, the discrete tokenization inherent in DLLMshinders the application of standard noise inversion techniques, often lead-ing to structural degradation during editing. In this paper, we introduceGIDE (Grounded Inversion for DLLM Image Editing), a unified frame-work designed to bridge this gap. GIDE incorporates a novel DiscreteNoise Inversion mechanism that accurately captures latent noise pat-terns within the discrete token space, ensuring high-fidelity reconstruc-tion. We then decompose the editing pipeline into grounding, inver-sion, and refinement stages. This design enables GIDE supportingvarious editing instructions (text, point and box) and operations whilestrictly preserving the unedited background. Furthermore, to overcomethe limitations of existing single-step evaluation protocols, we introduceGIDE-Bench, a rigorous benchmark comprising 805 compositional edit-ing scenarios guided by diverse multi-modal inputs. Extensive exper-iments on GIDE-Bench demonstrate that GIDE significantly outper-forms prior training-free methods, improving Semantic Correctness by51.83% and Perceptual Quality by 50.39%. Additional evaluations onImgEdit-Bench confirm its broad applicability, demonstrating consistentgains over trained baselines and yielding photorealistic consistency onpar with leading models.3 .
Show more
SelfMOTR: Revisiting MOTR with Self-Generating Detection Priors
Fabian Gülhan ⋅ Emil Mededovic ⋅ Yuli Wu ⋅ Johannes Stegmaier
End-to-end transformer architectures have driven significant progress in multi-object tracking by unifying detection and association into a single, heuristic-free framework. Despite these benefits, poor detection performance and the inherent conflict between detection and association in a joint architecture remain critical concerns. Recent approaches aim to mitigate these issues by employing advanced denoising or label assignment strategies, or by incorporating detection priors from external object detectors. In this paper, we propose SelfMOTR, a simple yet highly effective detector-free alternative that decouples proposal discovery from association using self-generated internal detection priors. Through extensive analysis and ablation studies, we show that end-to-end transformer trackers with joint detection–association decoding retain substantial hidden detection capacity, and we provide a practical detector-free mechanism for leveraging it. To shed light on these joint decoding dynamics, we draw inspiration from attention sink analyses in large language models, leveraging Track Attention Mass to show that standard generic queries exhibit unbalanced attention, frequently struggling to weigh track context against novel object discovery. SelfMOTR achieves highly competitive performance in complex, dynamic environments, yielding 69.2 HOTA on DanceTrack and leading with 71.1 HOTA on the Bird Flock Tracking (BFT) dataset. Project page: https://medem23.github.io/SM.
Show more
Push–Pull Attentional Anchoring for Diffusion Concept Erasure
Nattanat Chatthee ⋅ Tagon Sompong ⋅ Ekapol Chuangsuwanich ⋅ Supasorn Suwajanakorn
Rapid advances in diffusion models have raised concernsabout privacy, harmful content, and copyright violations. Concept era-sure methods aim to address these risks by removing undesirable con-cepts from pretrained models without full retraining. However, existingapproaches often degrade non-target generation quality through heuristicstrength scaling (e.g., classifier-free guidance or negative prompt scaling),leading to uncontrolled and excessive semantic drift. To address this, wepropose Push–Pull Attentional Anchoring (PPAA), a mechanism in thecross-attention space that displaces target representations while anchor-ing the scale of erasure via bounded cosine-similarity constraints. Byusing a relative similarity ratio instead of an absolute difference, ouradaptive threshold represents a meaningful percentage of semantic re-tention, facilitating a single value to be applied uniformly across layers.We conduct extensive experiments across multiple concept categories, in-cluding celebrities, artistic styles, nudity, and intellectual property. Ourmethod achieves favorable trade-offs against prior approaches on StableDiffusion v1.4 and demonstrates its applicability to modern architecturessuch as FLUX.
Show more
When Rubrics Fail: Error Enumeration as Reward for Reference-Free RL Post-Training
Wisdom Ikezogwo ⋅ Mehmet Saygin Seyfioglu ⋅ Ranjay Krishna ⋅ Karim Bouyarmane
Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers. But many real-world tasks admit multiple valid outputs and lack the single ideal answer that rubric generation depends on. We identify this reference-free setting as a gap in current post-training methods and propose Implicit Error Counting (IEC) to fill it. Instead of checking what a response gets right against a rubric, IEC enumerates what it gets wrong, applying severity-weighted scores across task-relevant axes and converting them into calibrated per-aspect rewards. We show that naïve explicit enumeration is too noisy for stable optimization, and that two design choices: implicit score emission and group calibration are necessary to make error counting a reliable reward. As a case study, we validate IEC on virtual try-on (VTO), a domain that is simultaneously too constrained for holistic scoring and too permissive for rubric-based evaluation: subtle garment errors are unacceptable, yet many output variations are correct. We introduce Cascaded Error Counting (CEC) as an evaluation metric, which tracks human preferences well (60% top-1 vs. 30% others), and curate Mismatch-DressCode (MDressBench), a benchmark with maximal attribute mismatch to stress-test reward designs. On MDressBench, IEC outperforms RaR across all metrics (CEC: 5.31 vs. 5.60 on flat references; 5.20 vs. 5.53 on non-flat). On VITON-HD and DressCode, IEC matches or surpasses six baselines on 6 of 8 perceptual metrics. These results suggest that when ideal answers are unavailable, counting errors provide a stronger signal than constructing rubrics.
Show more
PhenoLeaf-TS: A Time-Series Benchmark for Leaf Instance Segmentation, Tracking, and Growth Stage Classification
Rijad Saric ⋅ Basim Azam ⋅ Sarmad Khan ⋅ Edhem Custovic
Image-based high-throughput plant phenotyping leveragesinstance-level understanding of individual leaves growth over experimen-tal time, yet existing datasets lack temporal depth and annotation con-sistency to jointly benchmark segmentation, tracking, and growth stageclassification. We introduce PhenoLeaf-TS, a time-series dataset of 17,082top-down RGB images spanning 21 Arabidopsis thaliana genotypes, to-talling 318 plant replicates, each annotated with colour-coded leaf in-stance masks that maintain consistent identity throughout the growthsequence. We define three benchmark computer vision tasks with stan-dardised protocols and evaluate 21 distinct models: 9 instance segmenta-tion architectures, 6 multi-object trackers, and 6 classification architec-tures. Mask R-CNN with ResNet-50 achieves 73.2 mAP for leaf segmen-tation, ByteTrack reaches 84.1% MOTA for tracking individual leaves insequences, and Swin-T achieves 91.7% accuracy for growth stage classifi-cation. Cross-dataset experiments on CVPPP and Komatsuna show thatfine-tuning from PhenoLeaf-TS weights yields up to +51 mAP over zero-shot transfer. The dataset, the evaluation code, and the trained modelsare publicly accessible at https://pisyntor.github.io/PhenoLeaf-TS.
Show more
Drift-AR: Single-Step Visual Autoregressive Generation via Anti-Symmetric Drifting
zhen zou ⋅ Xiaoxiao Ma ⋅ Mingde Yao ⋅ Jie Huang ⋅ Linjiang Huang ⋅ Feng Zhao
Autoregressive (AR)-Di_x001B_usion hybrid paradigms combine AR'sstructured semantic modeling with di_x001B_usion's high-_x001C_delity synthesis, yetsu_x001B_er from a dual speed bottleneck: the sequential AR stage and theiterative multi-step denoising of the di_x001B_usion vision decode stage. Exist-ing methods address each in isolation without a uni_x001C_ed principle design.We observe that the per-position prediction entropy of continuous-spaceAR models naturally encodes spatially varying generation uncertainty,which simultaneously governing draft prediction quality in the AR stageand re_x001D_ecting the corrective e_x001B_ort required by vision decoding stage,which is not fully explored before. Since entropy is inherently tied toboth bottlenecks, it serves as a natural unifying signal for joint accel-eration. In this work, we propose Drift-AR, which leverages entropysignal to accelerate both stages: 1) for AR acceleration, we introduceEntropy-Informed Speculative Decoding that align draft_x0015_target entropydistributions via a causal-normalized entropy loss, resolving the entropymismatch that causes excessive draft rejection; 2) for visual decoder ac-celeration, we reinterpret entropy as the physical variance of the initialstate for an anti-symmetric drifting _x001C_eld_x0016_high-entropy positions acti-vate stronger drift toward the data manifold while low-entropy positionsyield vanishing drift_x0016_enabling single-step (1-NFE) decoding without it-erative denoising or distillation. Moreover, both stages share the sameentropy signal, which is computed once with no extra cost. Experimentson MAR, TransDi_x001B_, and NextStep-1 demonstrate 3.8_x0015_5.5× speedup withgenuine 1-NFE decoding, matching or surpassing original quality. Codeis available at https://github.com/aSleepyTree/Drift-AR.
Show more
Event-based Gaze Control Systems for Real-time Spin Estimation in Professional Ball Games
Yunpu Hu ⋅ Fabian Schilling ⋅ Valentina Cavinato ⋅ Asude Aydin ⋅ Agis Politis ⋅ Ricardo Morales ⋅ Kirk Scheper ⋅ Peter Dürr ⋅ Naoya Takahashi
Spin plays a crucial role in many ball sports due to its effecton the trajectory of the ball. Vision-based estimation of the ball’s spinduring a game with conventional cameras is challenging due to the ball’ssmall size, high speed, and fast rotation. To address these challenges, wepropose an event-based active vision system that can track unmodifiedballs and measure their spin in real time. The system consists of an eventcamera for its high temporal resolution and minimal motion blur, high-speed pan/tilt galvanometer mirrors to keep the ball in the field of view,and a low-latency focus-tunable telephoto lens to increase the spatial res-olution on the ball and keep it in focus. To track the ball, we use a hybridapproach that combines 2D event-based detection for centering and 3Dpositions from a ball localization system for re-initialization. For high-accuracy spin estimation, we propose an offline method that performscontrast maximization on the sphere (s-CMax). This method achievesstate-of-the-art accuracy on static balls across multiple sports (table ten-nis, baseball, tennis, and golf), with mean magnitude and axis errors of1.2% and 1.5 degrees, respectively. We then develop a low-latency onlinemethod for table tennis as a case study in real-time applications. Thismethod uses an uncertainty-aware convolutional neural network trainedon pseudo-ground-truth spin labels from the offline approach, combinedwith a GPU-accelerated batch implementation of contrast maximizationfor refinement. We demonstrate reliable tracking and spin estimationwith a three-view setup during professional table tennis matches, withhigh accuracy (8.8% magnitude and 6.4 degrees axis mismatch w.r.t. theoffline method), 3 ms latency, and 750 Hz throughput.
Show more
SPEAR: A Simulator for Photorealistic Embodied AI Research
Mike Roberts ⋅ Renhan Wang ⋅ Rushikesh Zawar ⋅ Rachith Dey-Prakash ⋅ Quentin Leboutet ⋅ Stephan Richter ⋅ Matthias Müller ⋅ German Ros ⋅ Rui Tang ⋅ Stefan Leutenegger ⋅ Yannick Hold-Geoffroy ⋅ Kalyan Sunkavalli ⋅ Vladlen Koltun
Interactive simulators have become powerful tools for train-ing embodied agents and generating synthetic visual data, but existingphotorealistic simulators suffer from limited generality, programmabil-ity, and rendering speed. We address these limitations by introducingSPEAR: A Simulator for Photorealistic Embodied AI Research. At itscore, SPEAR is a Python library that can connect to, and program-matically control, any Unreal Engine (UE) application via a modularplugin architecture. SPEAR exposes over 14K unique UE functions toPython, representing an order-of-magnitude increase in programmablefunctionality over existing UE-based simulators. Additionally, a singleSPEAR instance can render 1920×1080 photorealistic beauty images di-rectly into a user’s NumPy array at 73 frames per second – an order ofmagnitude faster than existing UE plugins – while also providing groundtruth image modalities that are not available in any existing UE-basedsimulator (e.g., a non-diffuse intrinsic image decomposition, material IDs,and physically based shading parameters). Finally, SPEAR introduces anexpressive high-level programming model that enables users to specifycomplex graphs of UE work with arbitrary data dependencies amongwork items, and to execute these graphs deterministically within a sin-gle UE frame. We demonstrate the utility of SPEAR through a diversecollection of example applications: controlling multiple embodied agentswith distinct action spaces (e.g., humans, cars, and robots) across sev-eral in-the-wild UE projects; rendering photorealistic city-scale environ-ments; manipulating UE’s procedural content generation systems; ren-dering synchronized multi-view images of detailed human faces; coordi-nating an interactive co-simulation with the MuJoCo physics simulator;and editing scenes with natural language via an AI coding assistant.
Show more
DoCoG: Mask-based Multi-Type Grounded Chain-of-Thought for Document QA
Sai Madhusudan Gunda ⋅ Jyothi Jinka ⋅ Hrithik Sagar ⋅ Aryan Jain ⋅ Venkata Venna ⋅ Anirudh Srinivasan ⋅ SANTOSH RAVI KIRAN SARVADEVABHATLA
Existing methods for Visual Document Question Answering(DocVQA) often predict answers without explicit intermediate analysisor produce ungrounded intermediate steps. The few approaches whichemploy grounding produce rectangular bounding boxes which fail to ac-curately represent complex structures such as pie-chart wedges, flowchartnodes, and curved text. We present DoCoG, a mask-based multi-typegrounded step-wise Chain-of-Thought architecture for Document QA.The model supports multi-type grounding, i.e., it localizes both textualcontent and fine-grained graphical elements such as legends, pie slices,and flowchart nodes. It generates step-wise intermediate outputs and fi-nal answers grounded through precise segmentation masks, enabling con-sistent evidence alignment. DoCoG integrates a Vision-Language Modelwith a promptable segmenter via a novel Grounding Interaction Module(GIM). During decoding, special [GND] tokens trigger mask prediction,linking each step and the final answer to supporting evidence. GIM re-fines [GND] tokens during decoding to improve spatial consistency andalignment. We introduce DoCoG-QA, a large-scale diverse dataset com-prising 325K documents and 1.5M step-wise grounded QA pairs. Wefurther release DoCoG-PQA with 20K preference pairs to improve align-ment using Direct Preference Optimization (DPO), and DoCoG-Bench,a human-verified benchmark. Experiments show that DoCoG achievesstate-of-the-art performance across multiple benchmarks, improving an-swer accuracy and grounding precision.
Show more
Predicting Consequences and Reinforcing Navigation Policies with Latent World Models
Zengmao Wang ⋅ Wei Gao ⋅ Shuhan Shen
World models enable agents to reason about future outcomesand learn policies from their knowledge of state transition, but existingapproaches primarily focus on reconstructing future observations or fea-tures, which introduces unnecessary complexity and limits their effec-tiveness for decision making. In this work, we propose a compatibilityprediction Latent World Model (LWM) for robot navigation that pre-dicts action-conditioned latent feature compatibility rather than recon-structing observations. Our key insight is that spatial proximity corre-lates with latent feature similarity, enabling action consequences to beevaluated directly in latent space. To support counterfactual training,our model leverages action sequences sampled across trajectories andlearns to predict which sequences lead closer to the goal. Furthermore,we demonstrate how the learned world model can supervise policy learn-ing from unlabeled video data and further improve policies through re-inforcement learning entirely within the world model. This imagination-driven framework eliminates the need for action annotations and addi-tional environment interaction. Extensive experiments on multiple real-world robot navigation datasets show that our approach significantlyoutperforms prior world model and imitation learning methods in pre-diction accuracy, policy learning, and real-world navigation performance.The code, pretrained models, and additional materials are available athttps://wzm206.github.io/latent-world-model-nav/.
Show more
SyntheticDoc: A Large Synthetic Dataset for Document Unwarping and Illumination Correction
Daniel Woortmann ⋅ Tanguy Magne ⋅ Olga Sorkine-Hornung
Deep learning models have become the standard tool for doc-ument rectification and illumination correction, yet their performance isfundamentally bound by their training data. For nearly a decade, the com-munity has heavily relied on Doc3D, a pioneering but increasingly limiteddocument unwarping dataset in terms of scale and quality. To address thisbottleneck, we introduce SyntheticDoc, a massive, high-quality datasetdesigned to push the boundaries of document unwarping. SyntheticDocis composed of 1,000,000 high-resolution procedurally generated trainingsamples, alongside extensive validation and test sets. Each sample ispaired with rich, pixel-perfect annotations, including UV maps, normalmaps, albedo and shading. To ensure physical accuracy and photorealism,the paper geometries are generated via a physics-based simulator andrendered using a path tracer. To demonstrate the benefit of our dataset,we train a simple baseline model on SyntheticDoc and report on itsperformance in comparison to state-of-the-art methods on both documentunwarping and illumination correction tasks. Our dataset is available athttps://igl.ethz.ch/projects/SyntheticDoc/ and the code used to generateit at https://github.com/tanguymagne/SyntheticDoc.
Show more
Unordered Landmark Visual Navigation
Hao Ren ⋅ Junzhe Zhu ⋅ Yihan Li ⋅ Zetong Bi ⋅ Le Zheng ⋅ Zhi Li ⋅ Yiqing Yuan ⋅ Zhaoliang Wan ⋅ Dizhe Zhang ⋅ Lu Qi ⋅ HUI CHENG
Image-goal navigation is a fundamental capability for em-bodied AI, yet its practical deployment is strained by strong prior as-sumptions. Existing methods predominantly rely on temporally orderedvideo streams or auxiliary sensors (e.g., depth, LiDAR) to maintain spa-tial consistency. These sequential and multimodal dependencies severelyrestrict scalability, especially when deploying robots using crowd-sourcedor pre-recorded unordered image collections. When temporal priors areremoved, current methods struggle with severe perceptual aliasing, noisyassociations, and catastrophic mapping failures. To address this under-explored challenge, we propose Unordered Landmark Visual Navigation(ULVN), a unified RGB-only framework free from temporal and odo-metric priors. ULVN systematically mitigates error accumulation by in-tegrating mapping, localization, and planning. Specifically, it constructsa robust 2D topological map directly from unstructured images via cali-brated geometric verification and maximum spanning forest refinement.For closed-loop execution, ULVN abandons sequential heuristics, utiliz-ing a graph-based belief propagation filter with entropy-adaptive fusionfor global localization and dynamic subgoal planning. Extensive experi-ments in simulation and real-world deployments demonstrate that ULVNsignificantly outperforms state-of-the-art methods.
Show more
PanoRec: Spatially-Structured Sequence Modeling for Multi-Granularity Panoramic Retrieval
Zidong Cao ⋅ Ding Zhou ⋅ Wenyao Gao ⋅ Lutao Jiang ⋅ Hui Xiong
Panoramic images capture holistic environments, yet retriev-ing them using fine-grained, localized textual descriptions remains a fun-damental challenge. Through empirical analysis, we reveal that modernVision-Language Models (VLMs) suffer from severe semantic dilutionwhen processing panoramic inputs. By compressing an information-densepanorama into a single global embedding, VLMs inevitably submerge lo-cal details within vast backgrounds, restricting retrieval to coarse scene-level matching. To overcome this bottleneck, we propose PanoRec, amulti-granularity panoramic retrieval framework built on a spatially-structured sequence modeling paradigm. Specifically, PanoRec serializesdistortion-free cubemap faces and a downsampled global panorama along-side spatial anchor tokens into a unified sequence. This enables the ef-ficient extraction of decoupled local and global representations within asingle forward pass. To effectively supervise this multi-granularity featurespace, we formulate a joint spatial InfoNCE objective. For local match-ing, we adopt a MaxSim routing strategy that dynamically aligns eachquery with its most relevant cubemap face. Crucially, this strategy notonly suppresses background noise during inference but also inherentlyintroduces a powerful hard spatial negative mining mechanism duringtraining. Extensive experiments demonstrate that PanoRec achieves im-pressive performance across multiple scenarios, effectively unifying holis-tic scene-level retrieval with fine-grained spatial discrimination.
Show more
R3DP: Real-Time 3D-Aware Policy for Embodied Manipulation
Yuhao Zhang ⋅ Wanxi Dong ⋅ Yue Shi ⋅ Yi Liang ⋅ Jingnan Gao ⋅ Qiaochu Yang ⋅ Yaxing Lyu ⋅ Zhixuan Liang ⋅ Yibin Liu ⋅ Congsheng Xu ⋅ Xianda Guo ⋅ Wei Sui ⋅ Yaohui Jin ⋅ Xiaokang Yang ⋅ Yanyan Xu ⋅ Yao Mu
Embodied manipulation requires accurate 3D understanding of objects and their spatial relations to plan and execute contactrich actions. While large-scale 3D foundation models provide strong priors, their computational cost incurs prohibitive latency for real-time control. We propose Real-time 3D-aware Policy (R3DP), which integrates powerful 3D priors into manipulation policies without sacrificing real-time performance. A core innovation of R3DP is the asynchronous fast–slow collaboration module, which sophisticatedly integrates largescale 3D model’s priors into the policy without compromising real-time performance. The system queries the pre-trained slow system (VGGT) only on sparse key frames, while simultaneously employing a lightweight Temporal Feature Prediction Network (TFPNet) to predict features for all intermediate frames. By leveraging historical data to exploit temporal correlations, TFPNet explicitly improves task success rates through consistent feature estimation. Additionally, we introduce a Multi-View Feature Fuser (MVFF) that aggregates features across views by explicitly incorporating camera intrinsics and extrinsics. R3DP offers a plug-and-play solution for integrating 3D foundation models into realtime inference systems. We evaluate R3DP against multiple baselines across different visual configurations. R3DP effectively harnesses largescale 3D priors to achieve superior results, outperforming single-view and multi-view DP by 32.9% and 51.4% in average success rates. Furthermore, by decoupling heavy 3D understanding from policy execution, R3DP achieves a 44.8% reduction in inference time compared to a naive DP+VGGT integration. Project page and code are available at: https://dazazh.github.io/r3dp-project-page/
Show more
NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding
Hyeonjeong Ha ⋅ Jinjin Ge ⋅ Bo Feng ⋅ Kaixin Ma ⋅ Gargi Chakraborty
Multimodal large language models (MLLMs) have achieved impressiveprogress in vision-language reasoning, yet their ability to understand temporallyunfolding narratives in videos remains largely underexplored. Narrative under-standing requires more than recognizing isolated events: models must maintaincoherent representations of who is doing what, when, and where across scene tran-sitions and temporal gaps. We introduce NARRATIVE T RACK, the first benchmarkto evaluate narrative understanding in MLLMs through fine-grained entity-centricreasoning. Unlike existing benchmarks limited to short clips or coarse scene-levelsemantics, we decompose videos into constituent entities and evaluate modelsusing a Compositional Reasoning Progression (CRP), a structured frameworkthat progressively increases narrative complexity across three dimensions: en-tity existence, entity changes, and entity ambiguity. This progression requiresmodels to move beyond local perception to reasoning about entities’ temporalpersistence, state changes, and fine-grained perceptual disambiguation. To enablescalable benchmark construction, we develop a fully automated entity-centricpipeline that extracts temporally grounded entity representations and providesthe foundation for CRP. Evaluations of state-of-the-art MLLMs reveal that ex-isting models struggle to maintain coherent entity representations under visualtransitions and temporal dynamics. Open-source general-purpose MLLMs exhibitstrong perceptual grounding but weak temporal continuity, while video-specializedMLLMs capture temporal context yet frequently hallucinate entities’ contexts.These findings uncover a fundamental trade-off between perceptual grounding andtemporal reasoning, indicating that narrative understanding emerges only fromtheir integration. NARRATIVE T RACK provides the first systematic framework todiagnose and advance temporally grounded narrative comprehension in MLLMs.
Show more
Humanoid Whole-Body Manipulation via Active Spatial Brain and Generalizable Action Cerebellum
Zhizhao Liang ⋅ Yi-Lin Wei ⋅ Xuhang Chen ⋅ Mu Lin ⋅ Yi-Xiang He ⋅ Zhexi Luo ⋅ Jun-Hui Liu ⋅ Kun-Yu Lin ⋅ WEISHI ZHENG
In this paper, we explore spatial-aware humanoid whole-body manipulation task. Compared with tabletop settings, this taskposes two key challenges: 1) Spatial understanding is challenging in com-plex 3D environments with diverse spatial relations. 2) Action generationis difficult to generalize, as limited and costly real-robot data restrictsdata-driven models generalization. To address these challenges, we pro-pose a generalizable humanoid loco-manipulation framework that lever-ages the spatial perception and action generation capabilities of multi-agent large models. Specifically, our framework includes two components:Active Spatial Brain for active spatial perception and decision-making,and Generalizable Action Cerebellum for executable robot action genera-tion. The first component actively perceives the spatial scene and makesdecisions on task planning and subtask decomposition. The second com-ponent generates executable robot actions based on the decisions madeby the first module without needs of task-specific real robot data. Tobenchmark our framework, we design a set of spatial manipulation tasksfrom two perspectives: evaluating spatial perception and understand-ing, and assessing real-robot task performance. The results demonstratestrong performance on both aspects across diverse tasks and environ-ments.
Show more
Incentivizing Vision Language Models to Search for Long Video Question Answering
Harsh Goel ⋅ S P Sharan ⋅ Sahil Shah ⋅ Minkyu Choi ⋅ Joungbin An ⋅ Kristen Grauman ⋅ Sandeep Chinchali
We introduce VSeek, an agentic framework that transformslong-video question answering (LVQA) from a passive, single-pass per-ception task into a multi-turn retrieval process. VSeek utilizes a naturallanguage-driven search to identify relevant context within long videosand is post-trained with reinforcement learning (RL) to jointly formulatetargeted search queries and reason over retrieved clips for LVQA. WhileRL post-training has revolutionized reasoning in symbolic domains suchas mathematics and code, its application to long-video understandingremains hindered by a lack of verified rewards. To ensure that the re-trieved context is relevant, we propose a novel neuro-symbolic approachthat bridges open-ended natural language with discrete visual verifica-tion. Specifically, complex user queries are compiled into formal tempo-ral logic specifications for systematically decomposing natural languagequestions into a definitive checklist of required atomic visual primitives,such as key objects and activities, along with their temporal ordering.These systematically derived grounding events provide the critical feed-back signal for RL post-training, enabling dense, verifiable rewards basedon the successful retrieval of these specific visual elements rather thanrelying entirely on outcome-only answer accuracy. By explicitly optimiz-ing for this verifiable evidence-seeking behavior, VSeek improves Pass@1scores by up to 8% and Pass@4 scores by 15% on long-video understand-ing benchmarks compared to base models. We open-source our code athttps://utaustin-swarmlab.github.io/VSeek.
Show more
Physically Grounded 3D Generative Reconstruction under Hand Occlusion using Proprioception and Multi-Contact Touch
Gabriele Mario Caddeo ⋅ Pasquale Marra ⋅ Lorenzo Natale
We propose a multimodal, physically grounded approach formetric-scale amodal object reconstruction and pose estimation undersevere hand occlusion. Unlike prior occlusion-aware 3D generation meth-ods that rely only on vision, we leverage physical interaction signals:proprioception provides the posed hand geometry, and multi-contacttouch constrains where the object surface must lie, reducing ambigu-ity in occluded regions. We represent object structure as a pose-aware,camera-aligned signed distance field (SDF) and learn a compact latentspace with a Structure-VAE. In this latent space, we train a condi-tional flow-matching diffusion model, pretraining on vision-only imagesand finetuning on occluded manipulation scenes while conditioning onvisible RGB evidence, occluder/visibility masks, the hand latent rep-resentation, and tactile information. Crucially, we incorporate physics-based objectives and differentiable decoder-guidance during finetuningand inference to reduce hand–object interpenetration and to align thereconstructed surface with contact observations. Because our methodproduces a metric, physically consistent structure estimate, it integratesnaturally into existing two-stage reconstruction pipelines, where a down-stream module refines geometry and predicts appearance. Simulation ex-periments show that adding proprioception and touch substantially im-proves completion under occlusion and yields physically plausible recon-structions at correct real-world scale compared to vision-only baselines;we further validate transfer by deploying the model on a real humanoidrobot with an end-effector different from those used during training. Seehttps://github.com/hsp-iit/physical-generative-reconstruction
Show more
Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing
Sen Liang ⋅ Cong Wang ⋅ Zhentao Yu ⋅ Fengbin Guan ⋅ zhengguang zhou ⋅ Teng Hu ⋅ youliang zhang ⋅ Yuan Zhou ⋅ Xin Li ⋅ Qinglin Lu ⋅ Zhibo Chen
Existing instruction-based video editing datasets commonlyfocus on single-task appearance editing, failing to meet the complex cre-ative demands of real-world scenarios. To bridge this gap, we presentGoku, a large-scale dataset featuring 2 million high-quality, instruction-aligned video editing pairs, which is the first to extend task boundariesfrom basic appearance editing to multi-task and structural manipulations(e.g., precise control of subject movement). To tackle the data synthesischallenges inherent in these complex tasks, we design an efficient datasynthesis pipeline that decomposes complex edits into controllable sub-problems and introduce a progressive filtering system for data reliabil-ity throughout the whole process. Furthermore, we explore the optimalnetwork structures on Goku, and propose Goku-Edit. To deeply com-prehend complex editing instructions, Goku-Edit leverages an MLLMas its text encoder and adopts a decoupled dual-branch design: a dedi-cated mask branch handles structural control, freeing the main branchfor appearance rendering. A comprehensive video editing benchmark,Goku-Bench, is also proposed with 1,000 human-verified test cases and7 novel editing-specific metrics. Evaluated on Goku-Bench, Goku-Editobtains up to +8% improvement on other open-source models in termsof instruction following.
Show more
Cooking beyond Frames: A Stereo Event Camera Dataset in the Kitchen
Chengming Feng ⋅ Hesam Araghi ⋅ Liming Zheng ⋅ Julien Dupeyroux ⋅ Xucong Zhang ⋅ Jan van Gemert ⋅ Nergis Tomen
Event cameras, also known as neuromorphic cameras, havegained significant attention in recent years due to their high tempo-ral resolution, high dynamic range, and low power consumption. Whilemany studies and datasets in neuromorphic vision have focused on auto-motive and drone applications, human-centric daily-life scenarios remainlargely underrepresented, despite their importance for developing andbenchmarking event-based perception systems. Moreover, the few ex-isting event-based human activity datasets are typically recorded withscripted human actions, limiting their ability to capture natural hu-man behaviors. In this paper, we introduce EventKitchen, a large-scalestereo event camera benchmark dataset of human cooking activities inthe kitchen. EventKitchen is egocentrically collected from 10 participantsin 13 diverse kitchens, where the participants wear a helmet with multiplesensors and naturally perform cooking activities, without any scriptedactions. EventKitchen comprises 5.5 hours of stereo event recordingswith synchronized RGB, depth, and IMU data. We provide human an-notations for 10,762 action segments and 13,482 bounding boxes. Wetrain baseline models on EventKitchen to perform multiple event-basedtasks, including action recognition, object detection, and stereo depthestimation. By capturing natural, real-world human activities, Event-Kitchen establishes a challenging benchmark for neuromorphic visionbeyond autonomous driving. The dataset and toolkit are available athttps://chengmingf.github.io/EventKitchen.github.io/
Show more
HumanOmni-Speaker: Identifying Who said What and When
Detao Bai ⋅ Xihan Wei ⋅ Zhiheng Ma
While Omni-modal Large Language Models have made strides in joint sensory processing, they fundamentally struggle with a cornerstone of human interaction: deciphering complex, multi-person conversational dynamics to accurately answer “Who said what and when.” Current models suffer from an “illusion of competence”—they exploit visual biases in conventional benchmarks to bypass genuine cross-modal alignment, while relying on sparse, low-frame-rate visual sampling that destroys crucial high-frequency dynamics like lip movements. To address this limitation, we introduce Visual-Registered Speaker Diarization and Recognition (VR-SDR) and the HumanOmni-Speaker Benchmark. By strictly eliminating visual shortcuts, this rigorous paradigm demands true end-to-end spatio-temporal identity binding using only natural language queries. To overcome the underlying architectural perception gap, we propose HumanOmni-Speaker, powered by a Visual Delta Encoder. By sampling raw video at 25 fps and explicitly compressing inter-frame motion residuals into just 6 tokens per frame, it captures fine-grained visemes and speaker trajectories without triggering a catastrophic token explosion. Ultimately, HumanOmni-Speaker demonstrates strong multimodal synergy, natively enabling end-to-end lip-reading and high-precision spatial localization without intrusive cropping, and achieving superior performance across a wide spectrum of speaker-centric tasks.
Show more
TriO: Tri-Modal Unsupervised Occupancy World Model for Anything Perception
Quinlan Sykora ⋅ Sourav Biswas ⋅ Christopher Diehl ⋅ Andrew Cunningham ⋅ Thomas Gilles ⋅ Raquel Urtasun
We present TriO, a multi-modal unsupervised world modelthat predicts 4D occupancy, obstacle segmentation, flow and LiDAR.In contrast to prior work, TriO utilizes three distinct sensor modali-ties (camera, LiDAR, and RADAR) as both inputs and sources of self-supervision, eliminating the need for additional human annotations. Thanksto its novel supervision, the model is able to segment any occupancyfrom the drivable surface, overcoming the limitations of existing open-set methods in handling long-tail objects. TriO achieves state-of-the-artresults in multiple 3D and 4D tasks, including occupancy, flow, and Li-DAR prediction, as well as zero-shot road obstacle segmentation acrossmultiple datasets such as Argoverse 2, and Spotting the Unexpected.Fig. 1: We present TriO, an unsupervised tri-modal occupancy world model designedto perceive and forecast anything that could be considered path-blocking.
Show more
UniTac: A Unified Multimodal Model for Cross-Sensor Tactile Understanding and Generation
Jiahang Tu ⋅ Fengyu Yang ⋅ Chenyang Ma ⋅ Xihang Yu ⋅ Ziyao Zeng ⋅ Shaokai Wu ⋅ Hanbin Zhao ⋅ Zhi Tao ⋅ Chao Zhang ⋅ Hui Qian ⋅ Alex Wong
Unified multimodal models (UMMs) have shown great promise in integrating understanding and generation across diverse modalities. However, existing research rarely extends this paradigm to the tactile domain, where both object-level semantics and sensor-level configurations jointly determine the meaning of touch. To address this gap, we propose UniTac, the first UMM designed for tactile understanding and generation. UniTac models the tactile process as a transition from noncontact to contact, capturing the physical interaction between sensors and objects through a dual-level representation that encodes both sensor and object attributes. For tactile understanding, UniTac introduces two tasks, object property description and sensor identification, to enhance reasoning over physical and cross-sensor information. For tactile generation, we design a two-stage training paradigm consisting of reconstruction and alignment, together with a sensor-prior-based sampling strategy that simulates realistic tactile contact. Trained on large-scale multi-sensor datasets, UniTac achieves state-of-the-art performance in tactile understanding and generates realistic tactile signals across sensors.
Show more
VOCA: Visual Odometry with Codec Awareness
Nouri Alexander Hilscher ⋅ Mateo de Mayo ⋅ Dominik Muhle ⋅ Christoph Hermes ⋅ Daniel Cremers
Camera pose estimation from image streams is a critical com-ponent of spatial world models that integrate perception into planningand decision-making. Nearly all Visual Odometry (VO) and SimultaneousLocalization and Mapping (SLAM) systems have focused on datasetscontaining raw, uncompressed videos. Many working systems insteaduse ubiquitous hardware units to efficiently compress and decode videostreams, saving orders of magnitude in storage and bandwidth. However,this lossy compression introduces visual artifacts that hinder the per-formance of traditional tracking systems. We present VOCA, a causalstereo visual-odometry method that exploits codec information to im-prove tracking performance. We achieve state-of-the-art performance oncausal VO for relative trajectory error, efficiency, and absolute trajec-tory error on compressed streams. This work highlights the potential ofleveraging widely available video codec information for vision tasks.
Show more
See & Sniff: Learning Visuo-Olfactory Representations
Seongyu Kim ⋅ Seungwoo Lee ⋅ Hyeonggon Ryu ⋅ Joon Son Chung ⋅ Arda Senocak
While modern multimodal models integrate vision with lan-guage, audio, or touch, olfaction remains largely unexplored due to thelack of paired visuo-olfactory data. We introduce SmellNet-V, a scal-able visuo-olfactory dataset built on the insight that odor identity islargely invariant to visual transformations within a semantic category.This allows us to synthetically pair smell-only samples with semanticallyaligned in-the-wild web images, converting a unimodal olfactory datasetinto a cross-modal benchmark without costly co-collection. Building onthis dataset, we propose See & Sniff, a self-supervised framework thatlearns joint visuo–olfactory representations via dense local alignment andnaturally produces smell saliency maps for spatial grounding of odorsources. We further introduce pixel-level smell localization task and abenchmark for evaluation. Our method surpasses smell-only baselines by7% in smell classification from smell alone and generalizes to cross-modalretrieval and smell localization, establishing visuo-olfactory learning asa new direction in multimodal perception.
Show more
PriorEye: Geospatial Visual Priors for End-to-End Autonomous Driving
Kyuhwan Yeon ⋅ Benjamin Ramtoula ⋅ Daniele De Martini
Most end-to-end autonomous driving methods rely solely oninstantaneous sensor observations, limiting them to reactive behaviorwithout the anticipatory foresight human drivers employ through priorexperience. We introduce geospatial visual priors, street-level visual con-text anchored to the intended driving route, providing visual-spatialforesight independent of real-time sensors. We propose a memory aug-mentation module featuring a dual-memory architecture and an adap-tive memory gate, which can be easily integrated into existing end-to-end approaches. This design pairs a contextual memory for retrievedpriors with a persistent fallback memory, and dynamically regulatesthe influence of memories based on current state compatibility. Eval-uated on the NAVSIM-v2 benchmark, our approach consistently im-proves performance across diverse end-to-end baselines. Furthermore,because these priors are independent of onboard sensors, our method in-herently improves robustness against sensor corruption, while the dual-memory design ensures safe fallback when the retrieved priors them-selves become unreliable. Our project page is available at https://ori-mrg.github.io/PriorEye.
Show more
Bridging Vision and Language Concepts through Optimal Transport Semantic Flow
Chenyang Zhang ⋅ Anqi Dong ⋅ Guangming Zhu ⋅ Nuoye Xiong ⋅ Siyuan Wang ⋅ Lin Mei ⋅ Liang Zhang
Concept Bottleneck Models (CBMs) promise transparent rea-soning by predicting through human-interpretable concepts, yet theireffectiveness fundamentally depends on how well visual and textual rep-resentations are aligned or matched. Existing vision–language CBMs of-ten rely on pre-aligned encoders or global cosine similarity, which ob-scures fine-grained concept localization and fails to reflect true seman-tic geometry. In this work, we rethink concept alignment as dynamiccross-modal transport process instead of static projection and proposeOptimal Transport Flow Concept Bottleneck Model (OTF-CBM). Itfirst learns a data-driven semantic cost via Inverse Optimal Transport tomeasure cross-modal distances, and then performs unbalanced optimal-transport-based flow matching to model semantic transitions betweenvisual patches and textual concepts. With velocity-based concept activa-tion, OTF-CBM captures interpretable geometric relations without ODEintegration. Experiments further show that OTF-CBM achieves greatclassification accuracy and concept faithfulness, offering a new geomet-ric and dynamical perspective for interpretable cross-modal reasoning.Our code can be found at github.com/ChenyangZhang00/OTF-CBM.
Show more
Generative Lane Topology Reasoning via Autoregressive Model with Geometry Prior
Jiahui Fu ⋅ Zehao Huang ⋅ Han Li ⋅ Naiyan Wang ⋅ Si Liu
Lane topology reasoning aims to construct a lane graph fromonboard sensor observations. Existing methods follow a detection and as-sociation paradigm that treats each lane instance independently, leadingto geometric inconsistency at connected endpoints and incomplete graphsdue to visual occlusions. To address these issues, we propose TopoGPT,a generative framework that learns the geometry prior from typical lanegraph structures through autoregressive sequence modeling. Specifically,we construct a large-scale map dataset comprising 3.3M scenes. For eachlane graph, a lane tokenizer serializes it into discrete tokens, while a scenecontext encoder converts it into a rasterized image and extracts globalfeatures as scene tokens. We pre-train an autoregressive lane sequencetransformer via scene-conditioned next-token prediction, endowing themodel with the geometry prior over lane graph structures. Building uponthis prior, a perception adapter aligns BEV features from multi-view im-ages with the pre-trained scene condition, transferring the learned geom-etry prior to sensor-based lane graph prediction. On the OpenLane-V2benchmark, TopoGPT outperforms existing methods by an average of+6.4 on lane-level and +11.6 on point-level metrics, and produces geo-metrically consistent and structurally complete lane graphs. Our projectpage is available at https://buaa-colalab.github.io/topogpt_page.
Show more
CoverPrune: Coverage-Driven Token Pruning for 3D VLMs via Optimal Transport
Peng Ling ⋅ Yingda Yin ⋅ Lingting Zhu ⋅ Weikai Chen ⋅ Shengju Qian ⋅ Zeyu HU ⋅ Xin Wang ⋅ Wenming Yang
While 3D Vision-Language Models (3D VLMs) have demon-strated remarkable spatial reasoning capabilities, they suffer from mas-sive visual token counts that create severe computational bottlenecksduring inference. Existing token pruning methods primarily rely on di-versity-based selection, discarding similar tokens to maximize dispersion.However, in 3D environments, this approach frequently drops represen-tative prototype tokens in favor of outliers, breaking the multi-view con-sistencies and geometric structures essential for spatial reasoning. In thispaper, we propose a paradigm shift for 3D VLM token pruning: frommaximizing diversity to preserving visual evidence coverage. We intro-duce CoverPrune, a training-free framework that formulates inference-time token pruning as an Optimal Transport (OT) problem. To over-come the intractable combinatorial subset selection inherent in this for-mulation, we design the Feature-Spatial-Temporal (FST) transport costand target capacity, along with an efficient Spatial-Guided Greedy Selec-tion (SGS) algorithm to approximate the OT objective. Furthermore, wepropose CoverPrune-Lite, an accelerated variant utilizing spatially struc-tured local matching for minimal overhead. Extensive experiments acrossmultiple 3D visual-spatial reasoning benchmarks demonstrate that ourmethods achieve state-of-the-art token efficiency, maintaining robust rea-soning performance even under highly aggressive pruning budgets. Visitour project website at https://github.com/Brucess/CoverPrune.
Show more
MV2GF: Multi-view Pedestrian Detection with a Visual Geometric Foundation Model
Taiga Yamane ⋅ Satoshi Suzuki ⋅ Ryo Masumura ⋅ Shota Orihashi ⋅ Tomohiro Tanaka ⋅ Mana Ihori ⋅ Naoki Makishima
Multi-View Pedestrian Detection (MVPD) aims to detectpedestrians in the form of a bird’s eye view map from multi-view images.Recent MVPD methods adopt a unified framework that projects 2D im-age features into a 3D world space and aggregates them into a singlefeature. Although they are effective, they struggle to generalize to un-seen camera configurations during training due to two main issues. First,they are difficult to capture accurate visual geometry across views in un-seen camera configurations. Second, they make detection models highlydependent on distortion patterns during training arising from their im-age feature projection. To address these, we leverage a visual geometricfoundation model and propose MV2GF. This foundation model has ex-hibited strong generalization in capturing visual geometry across viewsand predicting accurate 3D attributes in diverse camera configurations.MV2GF fuses task-specific features with general-purpose geometric fea-tures extracted by the foundation model to effectively capture the visualgeometry even in unseen camera configurations. Furthermore, MV2GFprojects each pixel in the image features to an appropriate 3D locationusing 3D pointmaps predicted by the foundation model, preventing thedetection model from depending on distortion patterns during training.Our experiments demonstrate the effectiveness of leveraging a visual geo-metric foundation model for MVPD and that MV2GF generalizes betterthan existing methods.
Show more
When Sinks Help or Hurt: Unified Framework for Attention Sink in MLLMs
Jiho Choi ⋅ Jaemin Kim ⋅ JINHWI PARK ⋅ Seunghoon Hong ⋅ Sanghwan Kim
Attention sinks are defined as tokens that attract dispro-portionate attention. While these have been studied in single modalitytransformers, their cross-modal impact in Large Vision-Language Mod-els (LVLM) remains largely unexplored: are they redundant artifactsor essential global priors? This paper first categorizes visual sinks intotwo distinct categories: ViT-emerged sinks (V-sinks), which propagatefrom the vision encoder, and LLM-emerged sinks (L-sinks), which arisewithin deep LLM layers. Based on the new definition, our analysis re-veals a fundamental performance trade-off: while sinks effectively encodeglobal scene-level priors, their dominance can suppress the fine-grainedvisual evidence required for local perception. Furthermore, we identifyspecific functional layers where modulating these sinks most significantlyimpacts downstream performance. To leverage these insights, we proposeLayer-wise Sink Gating (LSG), a lightweight, plug-and-play module thatdynamically scales the attention contributions of V-sink and the restvisual tokens. LSG is trained via standard next-token prediction, re-quiring no task-specific supervision while keeping the LVLM backbonefrozen. In most layers, LSG yields improvements on representative mul-timodal benchmarks, effectively balancing global reasoning and preciselocal evidence.1
Show more
CooperScene: Multi-Modal Cooperative Autonomy Benchmark with C-V2X Communication Characterization
Bo Wu ⋅ Ruoshen Mo ⋅ Justin Yue ⋅ Yanyu Zhang ⋅ Janice Nguyen ⋅ Guoyuan Wu ⋅ Amit Roy-Chowdhury ⋅ Matthew Barth ⋅ Hang Qiu
Cellular vehicle-to-everything (C-V2X) enables cooperativeperception, prediction, and planning beyond the field of view of individualagents. However, existing datasets often overlook the complexities of real-world deployment, such as limited communication bandwidth and its dy-namics, heterogeneous sensing modalities, and scalability beyond a singlecooperative partner. In this paper, we introduce CooperScene, a high-fidelity cooperative autonomy dataset with real-world C-V2X communica-tion characterization. The dataset is organized into diverse scenes, includ-ing intersections, highway ramps, and parking lots. These scenes involvethree connected and autonomous vehicles (CAVs) and one infrastructureroadside unit (RSU), all equipped with multi-modal sensors and commer-cial off-the-shelf C-V2X communication radios. All scenes are annotatedwith globally consistent 3D labels at 10 Hz, totaling 344K objects across59K frames, underpinned by tight sensor- and agent-synchronization,centimeter-level localization and spatial alignment, precise cross-modalitycalibration, and 3GPP-standard-compliant C-V2X communication. Coop-erScene establishes a rigorous benchmark for evaluating multi-agentscaling and actual performance in real-world deployable settings. Projectwebsite for data and benchmark: https://cisl.ucr.edu/CooperScene.
Show more
How to Teach Large Multimodal Models New Skills
Zhen Zhu ⋅ Yiming Gong ⋅ Yao Xiao ⋅ Yaoyao Liu ⋅ Derek Hoiem
How can we teach large multimodal models (LMMs) newskills without erasing prior abilities? We study sequential fine-tuningon five target skills while monitoring general ability on eight held-outbenchmarks across three model families. Surprisingly, we find that per-formance lost on held-out tasks after fine-tuning on one skill can partlyrecover when the model is subsequently tuned on a different skill. Wetrace this behavior to a measurable shift in the output token distribu-tion, manifested through a simple counting-bias probe that shows theshift co-varies with forgetting. Guided by this insight, we identify twosimple, robust tuning recipes that learn strongly while limiting drift: (i)updating only the self-attention projection layers (SA Proj., ∆ learning+24.9 / ∆ held-out forgetting -0.6), and (ii) updating only the MLPGate&Up while freezing the Down projection (+30.5 / -2.1). Both sub-stantially outperform full-LLM tuning (+31.8 / -23.3) in the learning–forgetting trade-off. We also compare against common forgetting mitiga-tion methods—Learning without Forgetting (LwF), LoRA, Mixture-of-Experts, and weight-space interpolation (WiSE-FT)—and find that ourselective tuning recipes match or exceed their learning–stability balancewhile remaining simpler, requiring no replay, auxiliary parameters, orper-stage tuning. These results hold across LLaVA-OneVision, LLaVA-NeXT, and Qwen2.5-VL, confirming that the key to teaching LMMs newskills without forgetting lies in controlling output distribution shift bychoosing which components to tune. Code will be made available.
Show more
VIPS: Vehicle-Infrastructure Cooperative Planning Benchmark via Pseudo-Simulation
Hoonhee Cho ⋅ Jae-young Kang ⋅ Giwon Lee ⋅ Hyemin Yang ⋅ Heejun Park ⋅ KUK-JIN YOON
End-to-end autonomous driving in urban environments re-quires robust decision-making under partial observability and complexmulti-agent interactions. Severe occlusions and dense traffic at inter-sections limit the perception capability of single-agent systems, moti-vating recent efforts on Vehicle-to-Infrastructure (V2I) cooperation forperception and planning. However, existing evaluation protocols face afundamental trade-off: open-loop evaluation fails to capture error accu-mulation and recovery from deviations, while closed-loop evaluation iscostly, difficult to scale, and often relies on simulated environments thatmay suffer from domain gaps. To bridge this gap, we propose VIPS, abenchmark for cooperative autonomous driving in V2I settings based onpseudo-simulation. VIPS extends pseudo-simulation by integrating ve-hicle and infrastructure observations. This enables scalable yet realisticevaluation of robustness and error propagation without full simulation.We further present CoS-V2X, a cooperative planning framework basedon sparse representations. CoS-V2X models vehicle–infrastructure inter-actions using compact features for efficient communication and robustdecision-making under heterogeneous observations. Code and dataset areavailable at https://vips2026.github.io.
Show more
Event-Driven Video Generation
Chika Maduabuchi ⋅ Jindong Wang
Current text-to-video models can make individual frames look convincing while still getting simple interactions wrong: objects move before contact, an intended action is skipped, a placed object keeps drifting, or a support relation breaks. Our starting point is that standard frame-first denoising updates every latent region at every step, even when the prompt implies that only a local interaction should be active. We introduce Event-Driven Video Generation (EVD), a small DiT-compatible intervention that gives the sampler an explicit event signal. A lightweight head predicts token-level event activity; training losses tie that activity to latent state change; and event-gated sampling, with hysteresis and an early-step schedule, applies the update field mainly where an interaction is forming. On EVD-Bench, EVD improves human preference and VBench dynamics for state persistence, spatial accuracy, support relations, and contact stability, while keeping appearance quality comparable to the base model. The results suggest that a modest amount of event structure can correct several interaction failures that otherwise remain hidden behind good frame-level appearance. Project webpage: https://evd-project-website.pages.dev
Show more
Successful Page Load