HoloSound: Combining Speech and Sound Identification for Deaf or Hard of Hearing Users on a Head-mounted Display

被引：19

作者：

Guo, Ru ^{[1
]}

Yang, Yiru ^{[1
]}

Kuang, Johnson ^{[1
]}

Bin, Xue ^{[2
]}

Jain, Dhruv ^{[1
]}

Goodman, Steven ^{[3
]}

Findlater, Leah ^{[3
]}

Froehlich, Jon E. ^{[1
]}

机构：

[1] Comp Sci & Engn, Seattle, WA 98195 USA

[2] HCI Design, Seattle, WA USA

[3] Human Ctr Design & Engn, Seattle, WA USA

来源：

22ND INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY (ASSETS '20) | 2020年

基金：

美国国家科学基金会;

关键词：

Augmented reality; head-mounted display; deaf; hard of hearing; speech-transcription; real-time captioning; sound recognition; sound localization; sound awareness;

D O I：

10.1145/3373625.3418031

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

Head-mounted displays can provide private and glanceable speech and sound feedback to deaf and hard of hearing people, yet prior systems have largely focused on speech transcription. We introduce HoloSound, a HoloLens-based augmented reality (AR) prototype that uses deep learning to classify and visualize sound identity and location in addition to providing speech transcription. This poster paper presents a working proof-of-concept prototype, and discusses future opportunities for advancing AR-based sound awareness.

引用

页数：4

共 19 条

[1] Temporal and spatio-temporal vibrotactile displays for voice fundamental frequency: An initial evaluation of a new vibrotactile speech perception aid with normal-hearing and hearing-impaired individuals [J].

Auer, ET ;

Bernstein, LE ;

Coulter, DC .

JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 1998, 104 (04) :2477-2489

[2] A Personalizable Mobile Sound Detector App Design for Deaf and Hard-of-Hearing Users [J].

Bragg, Danielle ;

Huynh, Nicholas ;

Ladner, Richard E. .

ASSETS'16: PROCEEDINGS OF THE 18TH INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY, 2016, :3-13

[3] Deaf and Hard-of-hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies [J].

Findlater, Leah ;

Chinh, Bonnie ;

Jain, Dhruv ;

Froehlich, Jon ;

Kushalnagar, Raja ;

Lin, Angela Carey .

CHI 2019: PROCEEDINGS OF THE 2019 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2019,

[4]

Fonseca E., 2017, P 18 ISMIR C, P486

[5] Deaf, Hard of Hearing, and Hearing Perspectives on using Automatic Speech Recognition in Conversation [J].

Glasser, Abraham ;

Kushalnagar, Kesavan ;

Kushalnagar, Raja .

PROCEEDINGS OF THE 19TH INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY (ASSETS'17), 2017, :427-432

[6]

Goodman Steven, P SIGCHI C HUM FACT

[7]

Gorman BenjaminM., 2014, P 16 INT ACM SIGACCE, P337, DOI DOI 10.1145/2661334.2661410

[8] Lightweight and optimized sound source localization and tracking methods for open and closed microphone array configurations [J].

Grondin, Francois ;

Michaud, Francois .

ROBOTICS AND AUTONOMOUS SYSTEMS, 2019, 113 :63-80

[9]

Hershey S, 2017, INT CONF ACOUST SPEE, P131, DOI 10.1109/ICASSP.2017.7952132

[10] SoundWatch: Exploring Smartwatch-based Deep Learning Approaches to Support Sound Awareness for Deaf and Hard of Hearing Users [J].

Jain, Dhruv ;

Ngo, Hung ;

Patel, Pratyush ;

Goodman, Steven ;

Findlater, Leah ;

Froehlich, Jon .

22ND INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY (ASSETS '20), 2020,

← 1 2 →