TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text

被引:47
作者
Singh, Amanpreet [1 ]
Peng, Guan [1 ]
Toh, Mandy [1 ]
Huang, Jing [1 ]
Galuba, Wojciech [1 ]
Hassner, Tal [1 ]
机构
[1] Facebook AI Res, Menlo Pk, CA 94025 USA
来源
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, CVPR 2021 | 2021年
关键词
D O I
10.1109/CVPR46437.2021.00869
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
A crucial component for the scene text based reasoning required for TextVQA and TextCaps datasets involve detecting and recognizing text present in the images using an optical character recognition (OCR) system. The current systems are crippled by the unavailability of ground truth text annotations for these datasets as well as lack of scene text detection and recognition datasets on real images disallowing the progress in the field of OCR and evaluation of scene text based reasoning in isolation from OCR systems. In this work, we propose TextOCR, an arbitrary-shaped scene text detection and recognition with 900k annotated words collected on real images from TextVQA dataset. We show that current state-of-the-art text-recognition (OCR) models fail to perform well on TextOCR and that training on TextOCR helps achieve state-of-the-art performance on multiple other OCR datasets as well. We use a TextOCR trained OCR model to create PixelM4C model which can do scene text based reasoning on an image in an end-to-end fashion, allowing us to revisit several design choices to achieve new state-of-the-art performance on TextVQA dataset.
引用
收藏
页码:8798 / 8808
页数:11
相关论文
共 60 条
[1]  
[Anonymous], 2005, P ACL WORKSH INTR EX
[2]   What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis [J].
Baek, Jeonghun ;
Kim, Geewook ;
Lee, Junyeop ;
Park, Sungrae ;
Han, Dongyoon ;
Yun, Sangdoo ;
Oh, Seong Joon ;
Lee, Hwalsuk .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :4714-4722
[3]  
Bigham J. P., 2010, P 23ND ANN ACM S USE, P333
[4]   Scene Text Visual Question Answering [J].
Biten, Ali Furkan ;
Tito, Ruben ;
Mafla, Andres ;
Gomez, Lluis ;
Rusinol, Marcal ;
Valveny, Ernest ;
Jawahar, C. V. ;
Karatzas, Dimosthenis .
2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, :4290-4300
[5]   Rosetta: Large Scale System for Text Detection and Recognition in Images [J].
Borisyuk, Fedor ;
Gordo, Albert ;
Sivakumar, Viswanath .
KDD'18: PROCEEDINGS OF THE 24TH ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY & DATA MINING, 2018, :71-79
[6]  
Chaudhry R, 2020, IEEE WINT CONF APPL, P3501, DOI 10.1109/WACV45572.2020.9093269
[7]  
Chee Kheng Chng, 2019, 2019 International Conference on Document Analysis and Recognition (ICDAR). Proceedings, P1571, DOI 10.1109/ICDAR.2019.00252
[8]  
Chng Chee Kheng, 2019, Int. J. Document Analysis Recogn.
[9]   Transferring metallic nano-island on hydrogen passivated silicon surface for nano-electronics [J].
Deng, J. ;
Troadec, C. ;
Joachim, C. .
SEMICONDUCTOR NANOSTRUCTURES TOWARDS ELECTRONIC AND OPTOELECTRONIC DEVICE APPLICATIONS II (SYMPOSIUM K, E-MRS 2009 SPRING MEETING), 2009, 6
[10]  
Devlin J, 2018, ARXIV