Augmented Reality Based Interactive Cooking Guide

被引:7
作者
Majil, Isaias [1 ]
Yang, Mau-Tsuen [1 ]
Yang, Sophia [2 ]
机构
[1] Natl Dong Hwa Univ, Dept Comp Sci & Informat Engn, Hualien 974301, Taiwan
[2] Natl Tsing Hua Univ, Interdisciplinary Program Elect Engn & Comp Sci, Hsinchu 300044, Taiwan
关键词
augmented reality; Magic Leap One; smart kitchen; AR cooking;
D O I
10.3390/s22218290
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Cooking at home is a critical survival skill. We propose a new cooking assistance system in which a user only needs to wear an all-in-one augmented reality (AR) headset without having to install any external sensors or devices in the kitchen. Utilizing the built-in camera and cutting-edge computer vision (CV) technology, the user can direct the AR headset to recognize available food ingredients by simply looking at them. Based on the types of the recognized food ingredients, suitable recipes are suggested accordingly. A step-by-step video tutorial providing details of the selected recipe is then displayed with the AR glasses. The user can conveniently interact with the proposed system using eight kinds of natural hand gestures without needing to touch any devices throughout the entire cooking process. Compared with the deep learning models ResNet and ResNeXt, experimental results show that the YOLOv5 achieves lower accuracy for ingredient recognition, but it can locate and classify multiple ingredients in one shot and make the scanning process easier for users. Twenty participants test the prototype system and provide feedback via two questionnaires. Based on the analysis results, 19 of the 20 participants would recommend others to use the proposed system, and all participants are overall satisfied with the prototype system.
引用
收藏
页数:18
相关论文
共 28 条
[1]  
[Anonymous], Q 100 INGREDIENTS CL
[2]  
[Anonymous], MAGIC LEAP GET START
[3]  
[Anonymous], Microsoft hololens
[4]  
[Anonymous], 2005, P C HUM FACT COMP SY
[5]  
Balaji A., 2020, P INT C SMART ELECT
[6]   Augmented/mixed reality technologies for food: A review [J].
Chai, Jackey J. K. ;
O'Sullivan, Carol ;
Gowen, Aoife A. ;
Rooney, Brendan ;
Xu, Jun-Li .
TRENDS IN FOOD SCIENCE & TECHNOLOGY, 2022, 124 :182-194
[7]  
Dormehl L., 2020, SAMSUNGS NEW FOOD CA
[8]  
Hasada H., 2019, INT C HUMAN COMPUTER
[9]  
Hashimoto A., 2008, P IPMU, V8, P848
[10]   Deep Residual Learning for Image Recognition [J].
He, Kaiming ;
Zhang, Xiangyu ;
Ren, Shaoqing ;
Sun, Jian .
2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778