If It's Not Efficient, It's Not Usable: Real-Time OOD Detection with Latent De-Biasing and High-Quality Negative Samples

On-device AI for resource-constrained systems, such as wear-ables and IoT sensors, requires models that are both highly efficientand reliable for mission-critical applications. Out-of-Distribution (OOD)detection is vital for ensuring this reliability, yet mainstream OOD re-search has focused on achieving state-of-the-art (SOTA) accuracy withoutregard for inference-time computational cost. This has led to powerful butcomplex methods (e.g., CLIP-based or supervised learning-based models)that are orders of magnitude too slow for real-world on-device deployment.We argue that the more meaningful benchmark for this domain is maxi-mizing accuracy within a practical, constrained inference computationalbudget. However, reducing inference complexity is typically achieved atthe expense of lower performance, exhibiting a trade-off between inferencespeed and accuracy. This paper breaks that trade-off. We introduce anovel framework that establishes an accuracy-SOTA under constrainedinference complexity. This is achieved through two key innovations: (i)a latent-space de-biasing technique that learns to remove pixel-intensitybias during training, drastically reducing inference complexity, and (ii) anintelligent negative sampling strategy that generates high-quality negativesamples to significantly boost performance. On both a real edge deviceand standard GPU infrastructure, extensive experiments reveal thatour method surpasses existing accuracy-SOTA OOD detection methodswithin a TinyML-appropriate inference budget.