Automatic Recognition and Augmentation of Attended Objects in Real-time using Eye Tracking and a Head-mounted Display

被引：11
|
作者：
Barz, Michael [1 ]
Kapp, Sebastian [2 ]
Kuhn, Jochen [2 ]
Sonntag, Daniel [1 ]
机构：
[1] Oldenburg Univ, German Res Ctr Artificial Intelligence DFKI, Oldenburg, Germany
[2] Tech Univ Kaiserslautern, Kaiserslautern, Rlp, Germany
来源：
ACM SYMPOSIUM ON EYE TRACKING RESEARCH AND APPLICATIONS (ETRA 2021) | 2021年
关键词：
eye tracking; augmented reality; visual attention; cognition-aware computing; computer vision;
D O I：
10.1145/3450341.3458766
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
Scanning and processing visual stimuli in a scene is essential for the human brain to make situation-aware decisions. Adding the ability to observe the scanning behavior and scene processing to intelligent mobile user interfaces can facilitate a new class of cognition-aware user interfaces. As a first step in this direction, we implement an augmented reality (AR) system that classifies objects at the user's point of regard, detects visual attention to them, and augments the real objects with virtual labels that stick to the objects in real-time. We use a head-mounted AR device (Microsoft HoloLens 2) with integrated eye tracking capabilities and a front-facing camera for implementing our prototype.
引用
收藏
页数：4
相关论文
共 47 条
  • [21] Real-time navigation guidance with intraoperative CT imaging for pedicle screw placement using an augmented reality head-mounted display: a proof-of-concept study
    Yanni, Daniel S.
    Ozgur, Burak M.
    Louis, Robert G.
    Shekhtman, Yevgenia
    Iyer, Rajiv R.
    Boddapati, Venkat
    Iyer, Asha
    Patel, Purvee D.
    Jani, Raja
    Cummock, Matthew
    Herur-Raman, Aalap
    Dang, Phuong
    Goldstein, Ira M.
    Brant-Zawadzki, Michael
    Steineke, Thomas
    Lenke, Lawrence G.
    NEUROSURGICAL FOCUS, 2021, 51 (02) : 1 - 8
  • [22] MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects
    Runz, Martin
    Buffier, Maud
    Agapito, Lourdes
    PROCEEDINGS OF THE 2018 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR), 2018, : 10 - 20
  • [23] Real-time head tracking from the deformation of eye contours using a piecewise affine camera
    Colombo, C
    Del Bimbo, A
    PATTERN RECOGNITION LETTERS, 1999, 20 (07) : 721 - 730
  • [24] Automated Calibration for Optical See-Through Head Mounted Display Using Display Screen Space Based Eye Tracking
    Bin Tomi, Azfar
    Rambli, Dayang Rohaya Awang
    2016 3RD INTERNATIONAL CONFERENCE ON COMPUTER AND INFORMATION SCIENCES (ICCOINS), 2016, : 448 - 453
  • [25] Integrated Approach of Dynamic Human Eye Movement Recognition and Tracking in Real-time
    Xiahou, JianBing
    He, Hao
    Wei, Ke
    She, Yingying
    2016 INTERNATIONAL CONFERENCE ON VIRTUAL REALITY AND VISUALIZATION (ICVRV 2016), 2016, : 94 - 101
  • [26] Implementation of Real-time Computer-generated Integral Imaging System for Head Mounted Display using OpenGL Shading Language
    Shin, SeungSeob
    Yeom, Jiwoon
    Son, Yeseul
    Choi, Kwang-Soon
    2019 10TH INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION TECHNOLOGY CONVERGENCE (ICTC): ICT CONVERGENCE LEADING THE AUTONOMOUS FUTURE, 2019, : 232 - 236
  • [27] Stopping by looking: A driver-pedestrian interaction study in a coupled simulator using head-mounted displays with eye-tracking
    Mok, Chun Sang
    Bazilinskyy, Pavlo
    de Winter, Joost
    APPLIED ERGONOMICS, 2022, 105
  • [28] Real-time Measurement and Display System of 3D Sound Intensity Map using Optical See-Through Head Mounted Display
    Kataoka, Yuta
    Teraoka, Wataru
    Oikawa, Yasuhiro
    Ikeda, Yusuke
    SA'18: SIGGRAPH ASIA 2018 POSTERS, 2018,
  • [29] Real-Time Navigation with Guide Template for Pedicle Screw Placement Using an Augmented Reality Head-Mounted Device: A Proof-of-Concept Study
    Haowei Li
    Peihai Zhang
    Guangzhi Wang
    Huiting Liu
    Xuejun Yang
    Guihuai Wang
    Zhenxing Sun
    Indian Journal of Orthopaedics, 2023, 57 : 776 - 781
  • [30] Real-Time Navigation with Guide Template for Pedicle Screw Placement Using an Augmented Reality Head-Mounted Device: A Proof-of-Concept Study
    Li, Haowei
    Zhang, Peihai
    Wang, Guangzhi
    Liu, Huiting
    Yang, Xuejun
    Wang, Guihuai
    Sun, Zhenxing
    INDIAN JOURNAL OF ORTHOPAEDICS, 2023, 57 (05) : 776 - 781
12345