Automatic Recognition and Augmentation of Attended Objects in Real-time using Eye Tracking and a Head-mounted Display

被引：14

作者：

Barz, Michael ^{[1
]}

Kapp, Sebastian ^{[2
]}

Kuhn, Jochen ^{[2
]}

Sonntag, Daniel ^{[1
]}

机构：

[1] Oldenburg Univ, German Res Ctr Artificial Intelligence DFKI, Oldenburg, Germany

[2] Tech Univ Kaiserslautern, Kaiserslautern, Rlp, Germany

来源：

ACM SYMPOSIUM ON EYE TRACKING RESEARCH AND APPLICATIONS (ETRA 2021) | 2021年

关键词：

eye tracking; augmented reality; visual attention; cognition-aware computing; computer vision;

D O I：

10.1145/3450341.3458766

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Scanning and processing visual stimuli in a scene is essential for the human brain to make situation-aware decisions. Adding the ability to observe the scanning behavior and scene processing to intelligent mobile user interfaces can facilitate a new class of cognition-aware user interfaces. As a first step in this direction, we implement an augmented reality (AR) system that classifies objects at the user's point of regard, detects visual attention to them, and augments the real objects with virtual labels that stick to the objects in real-time. We use a head-mounted AR device (Microsoft HoloLens 2) with integrated eye tracking capabilities and a front-facing camera for implementing our prototype.

引用

页数：4

共 27 条

[1] A systematic review of eye tracking research on multimedia learning [J].

Alemdag, Ecenaz ;

Cagiltay, Kursat .

COMPUTERS & EDUCATION, 2018, 125 :413-428

[2] Visual Search Target Inference in Natural Interaction Settings with Machine Learning [J].

Barz, Michael ;

Stauden, Sven ;

Sonntag, Daniel .

ETRA'20 FULL PAPERS: ACM SYMPOSIUM ON EYE TRACKING RESEARCH AND APPLICATIONS, 2020,

[3] Gaze-guided Object Classification using Deep Neural Networks for Attention-based Computing [J].

Barz, Michael ;

Sonntag, Daniel .

UBICOMP'16 ADJUNCT: PROCEEDINGS OF THE 2016 ACM INTERNATIONAL JOINT CONFERENCE ON PERVASIVE AND UBIQUITOUS COMPUTING, 2016, :253-256

[4] Evaluating Remote and Head-worn Eye Trackers in Multi-modal Speech-based HRI [J].

Barz, Michael ;

Poller, Peter ;

Sonntag, Daniel .

COMPANION OF THE 2017 ACM/IEEE INTERNATIONAL CONFERENCE ON HUMAN-ROBOT INTERACTION (HRI'17), 2017, :79-80

[5]

Blikstein P., 2013, P 3 INT C LEARN AN K, P102, DOI [DOI 10.1145/2460296.2460316, 10.1145/2460296.2460316]

[6] State-of-the-Art in Visual Attention Modeling [J].

Borji, Ali ;

Itti, Laurent .

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2013, 35 (01) :185-207

[7] Pervasive Attentive User Interfaces [J].

Bulling, Andreas .

COMPUTER, 2016, 49 (01) :94-98

[8] Comparing and Combining Interaction Data and Eye-tracking Data for the Real-time Prediction of User Cognitive Abilities in Visualization Tasks [J].

Conati, Cristina ;

Lalle, Sebastien ;

Rahman, Md Abed ;

Toker, Dereck .

ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS, 2020, 10 (02)

[9] Deep Residual Learning for Image Recognition [J].

He, Kaiming ;

Zhang, Xiangyu ;

Ren, Shaoqing ;

Sun, Jian .

2016 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2016, :770-778

[10]

Ijuin K., 2019, PROC 33 ANN C JAPANE

← 1 2 3 →