Event-based Gaze Control Systems for Real-time Spin Estimation in Professional Ball Games

Spin plays a crucial role in many ball sports due to its effecton the trajectory of the ball. Vision-based estimation of the ball’s spinduring a game with conventional cameras is challenging due to the ball’ssmall size, high speed, and fast rotation. To address these challenges, wepropose an event-based active vision system that can track unmodifiedballs and measure their spin in real time. The system consists of an eventcamera for its high temporal resolution and minimal motion blur, high-speed pan/tilt galvanometer mirrors to keep the ball in the field of view,and a low-latency focus-tunable telephoto lens to increase the spatial res-olution on the ball and keep it in focus. To track the ball, we use a hybridapproach that combines 2D event-based detection for centering and 3Dpositions from a ball localization system for re-initialization. For high-accuracy spin estimation, we propose an offline method that performscontrast maximization on the sphere (s-CMax). This method achievesstate-of-the-art accuracy on static balls across multiple sports (table ten-nis, baseball, tennis, and golf), with mean magnitude and axis errors of1.2% and 1.5 degrees, respectively. We then develop a low-latency onlinemethod for table tennis as a case study in real-time applications. Thismethod uses an uncertainty-aware convolutional neural network trainedon pseudo-ground-truth spin labels from the offline approach, combinedwith a GPU-accelerated batch implementation of contrast maximizationfor refinement. We demonstrate reliable tracking and spin estimationwith a three-view setup during professional table tennis matches, withhigh accuracy (8.8% magnitude and 6.4 degrees axis mismatch w.r.t. theoffline method), 3 ms latency, and 750 Hz throughput.