Tactile Modality Fusion for Vision-Language-Action Models

We propose TacFiLM, a lightweight modality-fusion ap-proach that integrates visual-tactile signals into vision-language-action(VLA) models. While advances in VLAs have introduced robot policiesthat are both generalizable and semantically grounded, these modelsmainly rely on vision-based perception. Vision alone, however, cannotcapture the complex interaction dynamics that occur during contact-richmanipulation, including contact forces, surface friction, compliance, andshear. While recent attempts to integrate tactile signals into VLA modelsoften increase complexity through token concatenation or large-scale pre-training, the heavy computational demands of behaviour models necessi-tate lightweight fusion strategies. To address these challenges, TacFiLMoutlines a post-training finetuning approach that conditions intermedi-ate visual features on pretrained tactile representations using feature-wise linear modulation (FiLM). Experimental results on insertion anddrawer opening tasks demonstrate consistent improvements in successrate, direct task performance, completion time, and force stability acrossboth in-distribution and out-of-distribution tasks. Together, these resultssupport our method as an effective approach to integrating tactile sig-nals into VLA models, improving contact-rich manipulation behaviours.Project page: https://charliem7.github.io/projects/TacFilm/