LiDAR-Event Stereo Fusion with Hallucinations

Event stereo matching is an emerging technique to estimate depth with event cameras. However, events themselves are unlikely to trigger in the absence of motion or the presence of large, untextured regions, making the correspondence problem much more challenging. Purposely, we propose integrating a stereo event camera with a fixed-frequency active sensor -- e.g., a LiDAR -- collecting sparse depth measurements, overcoming previous limitations while still enjoying the microsecond resolution of event cameras. Such depth hints are used to hallucinate events in the stacks or along the raw streams, compensating for the lack of information in the absence of brightness changes. Our techniques are general, can be adapted to any structured representation to stack events proposed in the literature, and outperform state-of-the-art fusion methods applied to event-based stereo.