ESTANet: Efficient Online Error Detection in Procedural Videos via Prediction Inconsistency

An efficient and accurate system for detecting errors in pro-cedural tasks is crucial for supporting human needs in daily life, asit can provide instant notifications and guide people to correct mis-takes. In this work, we study real-time online error detection in pro-cedural videos from a simple but overlooked perspective: the predictionbehavior of action detectors themselves. Instead of designing complexarchitectures or specialized supervision, we observe that action detec-tors naturally exhibit different prediction characteristics depending ontheir sensitivity to input dynamics and temporal context. We thereforepropose ESTANet (Error-Sensitive and Temporally-vArying Network),a lightweight framework that detects errors by exploiting inconsistenciesamong action predictions produced by a small set of action detectors.We construct standard and error-sensitive action detectors that behavesimilarly on correct executions but respond differently when errors occur.Meanwhile, detectors operating with different temporal contexts furtheramplify prediction inconsistencies when the procedure deviates from theintended sequence. During inference, we detect errors by aggregating mis-matches between standard and error-sensitive predictions through ma-jority voting to flag frames that contain errors. Extensive experimentson EgoPER, Assembly-101-O, and EPIC-Tent-O demonstrate that ES-TANet achieves state-of-the-art performance in online error detectionwhile maintaining real-time efficiency with a lightweight architecture.Our results highlight that leveraging the intrinsic properties of actiondetectors can yield a powerful and practical solution for online error de-tection without increasing architectural design complexity. Our code isavailable at: https://github.com/robert80203/ESTANet