A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments

被引：13

作者：

Jia, Jiyou ^{[1
]}

He, Yunfan ^{[1
]}

Le, Huixiao ^{[1
]}

机构：

[1] Peking Univ, Sch Educ, Dept Educ Technol, Yiheyuanlu 5, Beijing 100871, Peoples R China

来源：

BLENDED LEARNING: EDUCATION IN A SMART LEARNING ENVIRONMENT, ICBL 2020 | 2020年 / 12218卷

关键词：

Multimodal human-computer interaction; Intelligent tutoring system; Smart learning environment; Mathematics instruction; Learning of English as a foreign language; PEDAGOGICAL AGENT; INTERFACE AGENTS; ACHIEVEMENT;

D O I：

10.1007/978-3-030-51968-1_1

中图分类号：

TP31 [计算机软件];

学科分类号：

081202 ; 0835 ;

摘要：

A multimodal human-computer interaction system is composed of the comprehensive usage of various input and output channels. For the information input, apart from the traditional keyboard typing, mouse clicking, screen touching, the latest speech and face recognition technology can be used. For the output, the traditional screen display, the latest speech and facial expression synthesis and gesture generation can be used. After literature review of related works, this paper at first presents such a system, MMISE (Multimodal Interaction System for Education), about its architecture and working mechanism, POOOIIM (Pedagogical Objective Oriented Output, Input and Implementation Mechanism) illustrated with practical examples. Then this paper introduces this system's pilot applications in the epidemic time of novel coronavirus in 2020.

引用

页码：3 / 14

页数：12

共 13 条

[1] Multimodal human-computer interaction system for speech rehabilitation [J].

Zhao J. ;

Wang L. ;

Shi L.-J. ;

Kuang Z.-J. ;

Wang S.-B. ;

Sun Y.-Y. .

Jilin Daxue Xuebao (Gongxueban)/Journal of Jilin University (Engineering and Technology Edition), 2020, 50 (04) :1478-1486

[2] Multimodal human-computer interaction: A survey [J].

Jaimes, Alejandro ;

Sebe, Nicu .

COMPUTER VISION AND IMAGE UNDERSTANDING, 2007, 108 (1-2) :116-134

[3] Application of Speech Recognition Technology Based on Multimodal Information in Human-Computer Interaction [J].

Zhang, Yuan .

INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2024, 15 (09) :101-111

[4] Speech recognition and intelligent translation under multimodal human-computer interaction system [J].

Huang, Danhua ;

Xiang, Shuaiqiu .

JOURNAL OF INTELLIGENT SYSTEMS, 2024, 33 (01)

[5] Toward an affect-sensitive multimodal human-computer interaction [J].

Pantic, M ;

Rothkrantz, LJM .

PROCEEDINGS OF THE IEEE, 2003, 91 (09) :1370-1390

[6] A Study on Human-Computer Interaction Mechanism in an Intelligent Tutoring System [J].

Gao Hongli ;

Yang Lei ;

Xu Sheng ;

Long Zhou ;

Liu Kai ;

Hu Xiang'en .

FRONTIERS OF EDUCATION IN CHINA, 2022, 17 (01) :100-120

[7] Application of Virtual Reality Human-Computer Interaction Technology Based on the Sensor in English Teaching [J].

Jiang, Shuai ;

Wang, Lei ;

Dong, Yuanyuan .

JOURNAL OF SENSORS, 2021, 2021

[8] Predisposition-based intelligent tutoring system - Adaptive user profiling in human-computer interaction [J].

Niesler, Andrzej ;

Wydmuch, Gracja .

WEBIST 2008: PROCEEDINGS OF THE FOURTH INTERNATIONAL CONFERENCE ON WEB INFORMATION SYSTEMS AND TECHNOLOGIES, VOL 1, 2008, :435-440

[9] Development of adaptive human-computer interaction games to evaluate attention [J].

Kandemir, Hasan ;

Kose, Hatice .

ROBOTICA, 2022, 40 (01) :56-76

[10] Smart Groups: A system to orchestrate collaboration in hybrid learning environments. A simulation study [J].

Martin, Adrian Carruana ;

Alario-Hoyos, Carlos ;

Kloos, Carlos Delgado .

AUSTRALASIAN JOURNAL OF EDUCATIONAL TECHNOLOGY, 2022, 38 (06) :150-168

← 1 2 →