Beyond Script Family Boundaries: Towards Unified Open-Set Scene Text Recognition

Although VLMs are making fast progress in recognizing non-Latin scripts, they still face feasibility problems for less common scriptswith extremely low resources. Current Open-set text recognition methodsshow some good qualitative results, although they still lack quantitativeevidence. Specifically, the majority of open-set and zero-shot text recog-nition methods are confined to the CJK family, and to our knowledge,none handle more than one script family at the same time. In this work,we fill this gap with a unified, script-agnostic open-set text recognitionframework that can handle not just multiple scripts, but multiple scriptfamilies - Indic, Latin, and CJK scripts - in one set of partially sharedmodules. In this framework, we introduce a robust network structurethat can handle the complex character layout across a diverse collectionof scripts of different families. Furthermore, we introduce a heteroge-neous co-training method and tasks to enhance the model performanceon unseen scripts, and discuss the impacts from different data sourcesand task formats (character and word recognition). Experiments demon-strate that the proposed framework shows zero-shot learning capabilityon natural scene word images of Bengali, Gujarati, Japanese, Korean,and synthetic Yi samples. Specifically on the rare, yet actively used, Yiscript, our framework can achieve a word accuracy (ACR) of 16.10%,while the GPT5.2 shows an ACR of only 0, indicating a fundamentalcapability gap.