FindingDory: A Benchmark to Evaluate Memory in Embodied Agents

Rapid progress in embodied AI is bringing robots closer to performing complex tasks in real-world environments. In these settings, robots must operate over long time horizons, making decisions based on experiences collected over hours or even days. Vision-language models (VLMs) have recently shown strong capabilities in planning and control, making them promising high-level controllers for embodied agents. However, current VLMs can process only a limited number of images at once, highlighting the need for more efficient mechanisms to manage long-term memory in embodied contexts. To meaningfully evaluate these models for long-horizon control, a benchmark must target scenarios where memory is essential. Existing long-video QA benchmarks neglect embodied challenges like object manipulation and navigation, which require lowlevel skills and fine-grained reasoning over past interactions. Moreover, effective memory integration in embodied agents involves both recalling relevant historical information and executing actions based on that information, making it essential to study these aspects together. In this work, we introduce FindingDory, a new benchmark for long-range embodied tasks in the Habitat simulator. FindingDory evaluates memorycentric capabilities across 60 tasks requiring sustained engagement and contextual awareness in an environment. The tasks can also be procedurally extended to longer and more challenging versions, enabling scalable evaluation of memory and reasoning. We further present baselines that integrate state-of-the-art closed-source and fine-tuned open-source VLMs with low-level navigation policies, assessing their performance on these memory-intensive tasks and highlighting key areas for improvement.