On-Device Mobile Visual Location Recognition by Integrating Vision and Inertial Sensors

被引:68
作者
Guan, Tao [1 ]
He, Yunfeng [1 ]
Gao, Juan [1 ]
Yang, Jianzhong [1 ]
Yu, Junqing [1 ]
机构
[1] Huazhong Univ Sci & Technol, Sch Comp Sci & Technol, Wuhan 430074, Peoples R China
基金
中国国家自然科学基金;
关键词
Mobile visual location recognition; on-device; vector quantization; vision and inertial sensors integration;
D O I
10.1109/TMM.2013.2265674
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
This paper deals with the problem of city scale on-device mobile visual location recognition by fusing the inertial sensors and computer vision techniques. The main contributions are as follows: Firstly, we design an efficient vector quantization strategy by combining the Transform Coding (TC) and Residual Vector Quantization (RVQ). Our method can compress a visual descriptor into only several bytes while providing reasonable searching accuracy, which makes the managing of city scale image database directly on mobile devices come true. Secondly, we integrate the information from inertial sensors into the Vector of Locally Aggregated Descriptors (VLAD) generation and image similarity evaluation processes. Our method is not only fast enough for on-device implementation, but it also can improve the location recognition accuracy obviously. Thirdly, we also release a set of 1.295 million geo-tagged street view images with the information from inertial sensors, as well as a difficult set of query images. These resources can be used as a new benchmark to facilitate further research in the area. Experimental results prove the validity of the proposed methods for on-device mobile visual location recognition applications.
引用
收藏
页码:1688 / 1699
页数:12
相关论文
共 24 条
[11]   Histograms of oriented gradients for human detection [J].
Dalal, N ;
Triggs, B .
2005 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, VOL 1, PROCEEDINGS, 2005, :886-893
[12]   Aggregating Local Image Descriptors into Compact Codes [J].
Jegou, Herve ;
Perronnin, Florent ;
Douze, Matthijs ;
Sanchez, Jorge ;
Perez, Patrick ;
Schmid, Cordelia .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2012, 34 (09) :1704-1716
[13]   Product Quantization for Nearest Neighbor Search [J].
Jegou, Herve ;
Douze, Matthijs ;
Schmid, Cordelia .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2011, 33 (01) :117-128
[14]   Location Discriminative Vocabulary Coding for Mobile Landmark Search [J].
Ji, Rongrong ;
Duan, Ling-Yu ;
Chen, Jie ;
Yao, Hongxun ;
Yuan, Junsong ;
Rui, Yong ;
Gao, Wen .
INTERNATIONAL JOURNAL OF COMPUTER VISION, 2012, 96 (03) :290-314
[15]  
Ji RR, 2011, INT CONF ACOUST SPEE, P2400
[16]  
Kurz D., 2011, P IEEE C COMP VIS PA
[17]   Content and Context Boosting for Mobile Landmark Recognition [J].
Li, Zhen ;
Yap, Kim-Hui .
IEEE SIGNAL PROCESSING LETTERS, 2012, 19 (08) :459-462
[18]  
Nister D., 2006, 2006 IEEE COMP SOC C, V2, P2161, DOI [DOI 10.1109/CVPR.2006, 10.1109/CVPR.2006.264, DOI 10.1109/CVPR.2006.264]
[19]  
Schindler G., 2007, P IEEE C COMP VIS PA, V2007, P1
[20]  
Schroth G, 2012, INT CONF ACOUST SPEE, P2357, DOI 10.1109/ICASSP.2012.6288388