Teaching American Sign Language in Mixed Reality

被引：21

作者：

Shao, Qijia ^{[1
]}

Sniffen, Amy ^{[1
]}

Blanchet, Julien ^{[1
]}

Hillis, Megan E. ^{[2
]}

Shi, Xinyu ^{[3
]}

Haris, Themistoklis K. ^{[1
]}

Liu, Jason ^{[4
]}

Lamberton, Jason ^{[5
]}

Malzkuhn, Melissa ^{[5
]}

Quandt, Lorna C. ^{[6
]}

Mahoney, James ^{[1
]}

Kraemer, David J. M. ^{[4
]}

Zhou, Xia ^{[1
]}

Balkcom, Devin ^{[1
]}

机构：

[1] Dartmouth Coll, Dept Comp Sci, Hanover, NH 03755 USA

[2] Dartmouth Coll, Dept Psychol & Brain Sci, Hanover, NH 03755 USA

[3] Xiamen Univ, Dept Informat, Xiamen, Peoples R China

[4] Dartmouth Coll, Dept Educ, Hanover, NH 03755 USA

[5] Gallaudet Univ, Mot Light Lab, Washington, DC 20002 USA

[6] Gallaudet Univ, Educ Neurosci Program, Washington, DC 20002 USA

来源：

PROCEEDINGS OF THE ACM ON INTERACTIVE MOBILE WEARABLE AND UBIQUITOUS TECHNOLOGIES-IMWUT | 2020年 / 4卷 / 04期

基金：

美国国家科学基金会;

关键词：

Motion teaching; American Sign Language; Mixed reality;

D O I：

10.1145/3432211

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This paper presents a holistic system to scale up the teaching and learning of vocabulary words of American Sign Language (ASL). The system leverages the most recent mixed-reality technology to allow the user to perceive her own hands in an immersive learning environment with first- and third-person views for motion demonstration and practice. Precise motion sensing is used to record and evaluate motion, providing real-time feedback tailored to the specific learner. As part of this evaluation, learner motions are matched to features derived from the Hamburg Notation System (HNS) developed by sign-language linguists. We develop a prototype to evaluate the efficacy of mixed-reality-based interactive motion teaching. Results with 60 participants show a statistically significant improvement in learning ASL signs when using our system, in comparison to traditional desktop-based, non-interactive learning. We expect this approach to ultimately allow teaching and guided practice of thousands of signs.

引用

页数：27

共 50 条

[1] Mainepoint: Outcomes of teaching American sign language via distance learning
Keefe, B
Scherer, MJ
McKee, BG
TECHNOLOGY AND DISABILITY, 1996, 5 (3-4) : 319 - 326
[2] High Resolution 3D Models for the Teaching of American Sign Language
Odor, Alberto
ANNUAL REVIEW OF CYBERTHERAPY AND TELEMEDICINE, 2009, 7 : 82 - 86
[3] Learning sign language with mixed reality applications - the exploratory case study with deaf students
Imashev, Alfarabi
Kydyrbekova, Aigerim
Oralbayeva, Nurziya
Kenzhekhan, Azamat
Sandygulova, Anara
EDUCATION AND INFORMATION TECHNOLOGIES, 2024, 29 (13) : 17261 - 17292
[4] Holographic sign language avatar interpreter: A user interaction study in a mixed reality classroom
Yang, Fu-Chia
Mousas, Christos
Adamo, Nicoletta
COMPUTER ANIMATION AND VIRTUAL WORLDS, 2022, 33 (3-4)
[5] A quantitative analysis of sign lengthening in American Sign Language
Stewart, Jesse
SIGN LANGUAGE & LINGUISTICS, 2014, 17 (01) : 82 - 101
[6] Embodiment and American Sign Language Exploring sensory-motor influences in the recognition of American Sign Language
Corina, David P.
Gutierrez, Eva
GESTURE, 2016, 15 (03) : 291 - 305
[7] American Sign Language and early intervention
Snoddon, Kristin
CANADIAN MODERN LANGUAGE REVIEW-REVUE CANADIENNE DES LANGUES VIVANTES, 2008, 64 (04): : 581 - 604
[8] TOWARDS AN AMERICAN SIGN LANGUAGE INTERFACE
DORNER, B
HAGEN, E
ARTIFICIAL INTELLIGENCE REVIEW, 1994, 8 (2-3) : 235 - 253
[9] The landscape of American Sign Language education
Pirone, John S.
Pudans-Smith, Kimberly K.
Ivy, Tracy
Listman, Jason D.
FOREIGN LANGUAGE ANNALS, 2023, 56 (02) : 238 - 258
[10] The acquisition of prosody in American Sign Language
Brentari, Diane
Falk, Joshua
Wolford, George
LANGUAGE, 2015, 91 (03) : E144 - E168

← 1 2 3 4 5 →