Smartphone-based food recognition system using multiple deep CNN models

被引:31
作者
Fakhrou, Abdulnaser [1 ]
Kunhoth, Jayakanth [2 ]
Al Maadeed, Somaya [2 ]
机构
[1] Qatar Univ, Coll Educ, Dept Psychol Sci, Doha, Qatar
[2] Qatar Univ, Dept Comp Sci & Engn, Doha, Qatar
关键词
Food classification; Deep learning; Ensemble learning; Assistive system; Visual impairment;
D O I
10.1007/s11042-021-11329-6
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
People with blindness or low vision utilize mobile assistive tools for various applications such as object recognition, text recognition, etc. Most of the available applications are focused on recognizing generic objects. And they have not addressed the recognition of food dishes and fruit varieties. In this paper, we propose a smartphone-based system for recognizing the food dishes as well as fruits for children with visual impairments. The Smartphone application utilizes a trained deep CNN model for recognizing the food item from the real-time images. Furthermore, we develop a new deep convolutional neural network (CNN) model for food recognition using the fusion of two CNN architectures. The new deep CNN model is developed using the ensemble learning approach. The deep CNN food recognition model is trained on a customized food recognition dataset.The customized food recognition dataset consists of 29 varieties of food dishes and fruits. Moreover, we analyze the performance of multiple state of art deep CNN models for food recognition using the transfer learning approach. The ensemble model performed better than state of art CNN models and achieved a food recognition accuracy of 95.55 % in the customized food dataset. In addition to that, the proposed deep CNN model is evaluated in two publicly available food datasets to display its efficacy for food recognition tasks.
引用
收藏
页码:33011 / 33032
页数:22
相关论文
共 36 条
[21]   DietCam: Automatic dietary assessment with mobile camera phones [J].
Kong, Fanyu ;
Tan, Jindong .
PERVASIVE AND MOBILE COMPUTING, 2012, 8 (01) :147-163
[22]  
Lanigan PE, 2006, TENTH IEEE INTERNATIONAL SYMPOSIUM ON WEARABLE COMPUTERS, PROCEEDINGS, P147
[23]  
Liu C, 2018, IEEE T SERV COMPUT, V11, P249, DOI 10.1109/TSC.2017.2662008
[24]   Wide-Slice Residual Networks for Food Recognition [J].
Martinel, Niki ;
Foresti, Than Luca ;
Micheloni, Christian .
2018 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2018), 2018, :567-576
[25]  
Matsuda Y., 2012, 2012 IEEE International Conference on Multimedia and Expo (ICME), P25, DOI 10.1109/ICME.2012.157
[26]   Im2Calories: towards an automated mobile vision food diary [J].
Myers, Austin ;
Johnston, Nick ;
Rathod, Vivek ;
Korattikara, Anoop ;
Gorban, Alex ;
Silberman, Nathan ;
Guadarrama, Sergio ;
Papandreou, George ;
Huang, Jonathan ;
Murphy, Kevin .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :1233-1241
[27]   Smartphone-Based Escalator Recognition for the Visually Impaired [J].
Nakamura, Daiki ;
Takizawa, Hotaka ;
Aoyagi, Mayumi ;
Ezaki, Nobuo ;
Mizuno, Shinji .
SENSORS, 2017, 17 (05)
[28]   FoodNet: Recognizing Foods Using Ensemble of Deep Networks [J].
Pandey, Paritosh ;
Deepthi, Akella ;
Mandal, Bappaditya ;
Puhan, N. B. .
IEEE SIGNAL PROCESSING LETTERS, 2017, 24 (12) :1758-1762
[29]   Cloud-based SVM for food categorization [J].
Pouladzadeh, Parisa ;
Shirmohammadi, Shervin ;
Bakirov, Aslan ;
Bulut, Ahmet ;
Yassine, Abdulsalam .
MULTIMEDIA TOOLS AND APPLICATIONS, 2015, 74 (14) :5243-5260
[30]  
Qiu J., 2019, British Machine Vision Conference, P588, DOI [10.48550/arXiv.2207.03692, DOI 10.48550/ARXIV.2207.03692]