"Not There Yet": Feasibility and Challenges of Mobile Sound Recognition to Support Deaf and Hard-of-Hearing People

被引：4
作者：
Huang, Jeremy Zhengqi [1 ]
Chhabria, Hriday [1 ]
Jain, Dhruv [1 ]
机构：
[1] Univ Michigan, Ann Arbor, MI 48109 USA
来源：
PROCEEDINGS OF THE 25TH INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY, ASSETS 2023 | 2023年
关键词：
Accessibility; deaf; Deaf; hard of hearing; sound awareness; sound recognition; field study;
D O I：
10.1145/3597638.3608431
中图分类号：
TP301 [理论、方法];
学科分类号：
081202 ;
摘要：
While recent advances have enabled mobile sound recognition tools for deaf and hard of hearing (DHH) people, these tools have only been studied in the lab or through short, controlled experiments. To assess the real-world feasibility and guide the future designs of mobile sound awareness systems, we conducted a three-week field study of SoundWatch, a smartwatch-based sound recognition app, with 10 DHH participants. Our findings suggest the app's utility in increasing environmental awareness and facilitating everyday tasks for DHH users. However, several challenges, such as background noises, variability of real-world sounds, and confusion among similar sounding sounds, indicated that mobile sound recognition solutions are "not there yet" for adoption and use in daily life. We close by presenting HCI design opportunities to improve model reliability by increasing contextual awareness, supporting end-user customization, and fostering the collective improvement of sound recognition models.
引用
收藏
页数：14
相关论文
共 55 条
[21]   HoloSound: Combining Speech and Sound Identification for Deaf or Hard of Hearing Users on a Head-mounted Display [J].
Guo, Ru ;
Yang, Yiru ;
Kuang, Johnson ;
Bin, Xue ;
Jain, Dhruv ;
Goodman, Steven ;
Findlater, Leah ;
Froehlich, Jon E. .
22ND INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY (ASSETS '20), 2020,
[22]   Using a Participatory Activities Toolkit to Elicit Privacy Expectations of Adaptive Assistive Technologies [J].
Hamidi, Foad ;
Poneres, Kellie ;
Massey, Aaron ;
Hurst, Amy .
17TH INTERNATIONAL WEB FOR ALL CONFERENCE (WEB4ALL), 2020,
[23]   Who Should Have Access to my Pointing Data? Privacy Tradeoffs of Adaptive Assistive Technologies [J].
Hamidi, Foad ;
Poneres, Kellie ;
Massey, Aaron ;
Hurst, Amy .
ASSETS'18: PROCEEDINGS OF THE 20TH INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY, 2018, :203-216
[24]  
Ho-Ching F Wai-ling, Can you see what I hear? The Design and Evaluation of a Peripheral Sound Display for the Deaf
[25]   PrivacyMic: Utilizing Inaudible Frequencies for Privacy Preserving Daily Activity Recognition [J].
Iravantchi, Yasha ;
Ahuja, Karan ;
Goel, Mayank ;
Harrison, Chris ;
Sample, Alanson .
CHI '21: PROCEEDINGS OF THE 2021 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS, 2021,
[26]   ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard-of-Hearing Users [J].
Jain, Dhruv ;
Nguyen, Khoa Huynh Anh ;
Goodman, Steven ;
Grossman-Kahn, Rachel ;
Ngo, Hung ;
Kusupati, Aditya ;
Du, Ruofei ;
Olwal, Alex ;
Findlater, Leah ;
Froehlich, Jon E. .
PROCEEDINGS OF THE 2022 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI' 22), 2022,
[27]   SoundWatch: Exploring Smartwatch-based Deep Learning Approaches to Support Sound Awareness for Deaf and Hard of Hearing Users [J].
Jain, Dhruv ;
Ngo, Hung ;
Patel, Pratyush ;
Goodman, Steven ;
Findlater, Leah ;
Froehlich, Jon .
22ND INTERNATIONAL ACM SIGACCESS CONFERENCE ON COMPUTERS AND ACCESSIBILITY (ASSETS '20), 2020,
[28]   HomeSound: An Iterative Field Deployment of an In-Home Sound Awareness System for Deaf or Hard of Hearing Users [J].
Jain, Dhruv ;
Mack, Kelly ;
Amrous, Akli ;
Wright, Matt ;
Goodman, Steven ;
Findlater, Leah ;
Froehlich, Jon E. .
PROCEEDINGS OF THE 2020 CHI CONFERENCE ON HUMAN FACTORS IN COMPUTING SYSTEMS (CHI'20), 2020,
[29]   Framing reinforcement learning from human reward: Reward positivity, temporal discounting, episodicity, and performance [J].
Knox, W. Bradley ;
Stone, Peter .
ARTIFICIAL INTELLIGENCE, 2015, 225 :24-50
[30]  
Kulesza Todd, 2015, P 20 INT C INT US IN, P126, DOI DOI 10.1145/2678025.2701399
123456