A Multimodal Human-Computer Interaction System and Its Application in Smart Learning Environments

被引：12
|
作者：
Jia, Jiyou [1 ]
He, Yunfan [1 ]
Le, Huixiao [1 ]
机构：
[1] Peking Univ, Sch Educ, Dept Educ Technol, Yiheyuanlu 5, Beijing 100871, Peoples R China
来源：
BLENDED LEARNING: EDUCATION IN A SMART LEARNING ENVIRONMENT, ICBL 2020 | 2020年 / 12218卷
关键词：
Multimodal human-computer interaction; Intelligent tutoring system; Smart learning environment; Mathematics instruction; Learning of English as a foreign language; PEDAGOGICAL AGENT; INTERFACE AGENTS; ACHIEVEMENT;
D O I：
10.1007/978-3-030-51968-1_1
中图分类号：
TP31 [计算机软件];
学科分类号：
081202 ; 0835 ;
摘要：
A multimodal human-computer interaction system is composed of the comprehensive usage of various input and output channels. For the information input, apart from the traditional keyboard typing, mouse clicking, screen touching, the latest speech and face recognition technology can be used. For the output, the traditional screen display, the latest speech and facial expression synthesis and gesture generation can be used. After literature review of related works, this paper at first presents such a system, MMISE (Multimodal Interaction System for Education), about its architecture and working mechanism, POOOIIM (Pedagogical Objective Oriented Output, Input and Implementation Mechanism) illustrated with practical examples. Then this paper introduces this system's pilot applications in the epidemic time of novel coronavirus in 2020.
引用
收藏
页码：3 / 14
页数：12
相关论文
共 50 条
  • [1] A Multimodal Human-Computer Interaction for Smart Learning System
    Alzubi, Tareq Mahmod
    Alzubi, Jafar A.
    Singh, Ashish
    Alzubi, Omar A.
    Subramanian, Murali
    INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 2025, 41 (03) : 1718 - 1728
  • [2] Human-Computer Interaction in Smart Environments
    Paravati, Gianluca
    Gatteschi, Valentina
    SENSORS, 2015, 15 (08): : 19487 - 19494
  • [3] A Multimodal Human-Computer Interaction for Smart Learning System (May 2023, 10.1080/10447318.2023.2206758 )
    Alzubi, T. M.
    Alzubi, J. A.
    Singh, A.
    Alzubi, O. A.
    Subramanian, M.
    INTERNATIONAL JOURNAL OF HUMAN-COMPUTER INTERACTION, 2025, 41 (03) : 0I - 0I
  • [4] Physicians in virtual environments -: multimodal human-computer interaction
    Krapichler, C
    Haubner, M
    Lösch, A
    Schuhmann, D
    Seemann, M
    Englmeier, KH
    INTERACTING WITH COMPUTERS, 1999, 11 (04) : 427 - 452
  • [5] Multimodal human-computer interaction
    Turk, M
    REAL-TIME VISION FOR HUMAN-COMPUTER INTERACTION, 2005, : 269 - 283
  • [6] Multimodal human-computer interaction system for speech rehabilitation
    Zhao J.
    Wang L.
    Shi L.-J.
    Kuang Z.-J.
    Wang S.-B.
    Sun Y.-Y.
    Shi, Li-Juan (shilj@ccu.edu.cn), 1600, Editorial Board of Jilin University (50): : 1478 - 1486
  • [7] Service Design for Developing Multimodal Human-Computer Interaction for Smart Tvs
    Wang, Sheng-Ming
    Huang, Cheih-Ju
    INTERNATIONAL JOURNAL OF ADVANCED COMPUTER SCIENCE AND APPLICATIONS, 2015, 6 (02) : 227 - 234
  • [8] From human-computer interaction to human-artefact interaction:: Interaction design for smart environments
    Streitz, NA
    FROM INTEGRATED PUBLICATION AND INFORMATION SYSTEMS TO VIRTUAL INFORMATION AND KNOWLEDGE ENVIRONMENTS: ESSAYS DEDICATED TO ERICH J NEUHOLD ON THE OCCASION OF HIS 65TH BIRTHDAY, 2005, 3379 : 232 - 240
  • [9] Multimodal Interfaces of Human-Computer Interaction
    Karpov, A. A.
    Yusupov, R. M.
    HERALD OF THE RUSSIAN ACADEMY OF SCIENCES, 2018, 88 (01) : 67 - 74
  • [10] Considerations on Multimodal Human-Computer Interaction
    Liu, Chang
    Xie, Wenjun
    Zhang, Peng
    Zhan, Jia
    Xiao, Zonghao
    PROCEEDINGS OF 2018 5TH IEEE INTERNATIONAL CONFERENCE ON CLOUD COMPUTING AND INTELLIGENCE SYSTEMS (CCIS), 2018, : 331 - 335
12345