An Investigation of Speech Features, Plant System Alarms, and Operator-System Interaction for the Classification of Operator Cognitive Workload During Dynamic Work

被引:6
作者
Braarud, Per O. [1 ]
Bodal, Terje [1 ]
Hulsund, John E. [1 ]
Louka, Michael N. [1 ]
Nihlwing, Christer [1 ]
Nystad, Espen [1 ]
Svengren, Hakan [1 ]
Wingstedt, Emil [1 ]
机构
[1] Inst Energy Technol, POB 173, NO-1751 Halden, Norway
关键词
cognitive workload; dynamic measurement; nonintrusive; machine learning; simulator research; MENTAL WORKLOAD; HUMAN-PERFORMANCE; HEART-RATE; LOAD; FATIGUE; STRESS;
D O I
10.1177/0018720820961730
中图分类号
B84 [心理学]; C [社会科学总论]; Q98 [人类学];
学科分类号
03 ; 0303 ; 030303 ; 04 ; 0402 ;
摘要
Objective To investigate speech features, human-machine alarms, and operator-system interaction for the estimation of cognitive workload in full-scale realistic simulated scenarios. Background Theories and models of cognitive workload are critical for the design and evaluation of human-machine systems. Unfortunately, there are very few nonintrusive cognitive workload measures available for realistic dynamic human-machine interaction. Method The study was conducted in a full-scope control room research simulator of an advanced nuclear reactor. Six crews, each consisting of three operators, participated in 12 scenarios. The operators rated their workload every second minute. Machine learning algorithms were trained to estimate operators' workload based on crew communication, operator-system interaction, and system alarms. Results Random Forest (RF) utilizing speech and system features achieved an accuracy of 67% on test data. Utilizing speech features only, the accuracy achieved was 63%. The most important speech features were pitch, amplitude, and articulation rate. A 61% accuracy was achieved when alarms and operator-system interaction features were used. The most important features were the number of alarms and amount of operator-system interaction. Accuracy for algorithms trained for each operator ranged from 39% to 98%, with an average of 72%. For a majority of analyses performed, RF and extreme gradient boosting (XGB) outperformed other algorithms. Conclusion The results demonstrate that the features investigated and machine learning models developed provide a potential for the dynamic nonintrusive measurement of cognitive workload. Application The approach presented can be developed for nonintrusive workload measurement in real-world human-machine applications, simulator-based training, and research.
引用
收藏
页码:736 / 756
页数:21
相关论文
共 85 条
[1]   Using eye movement activity as a correlate of cognitive workload [J].
Ahlstrom, Ulf ;
Friedman-Berg, Ferne J. .
INTERNATIONAL JOURNAL OF INDUSTRIAL ERGONOMICS, 2006, 36 (07) :623-636
[2]  
[Anonymous], 2014, INT CONF ACOUST SPEE
[3]   Fixed point of a pair of contractive dominated mappings on a closed ball in an ordered dislocated metric space [J].
Arshad, Muhammad ;
Shoaib, Abdullah ;
Beg, Ismat .
FIXED POINT THEORY AND APPLICATIONS, 2013,
[4]  
Boersma P., 2001, Praat: Doing Phonetics by Computer, V5, P341, DOI DOI 10.1097/AUD.0B013E31821473F7
[5]  
Braarud P. O., 2020, 1261 OECD HALD REACT
[7]  
Braarud PO, 2011, SIMULATOR-BASED HUMAN FACTORS STUDIES ACROSS 25 YEARS: THE HISTORY OF THE HALDEN MAN-MACHINE LABORATORY, P233, DOI 10.1007/978-0-85729-003-8_15
[8]  
BRENNER M, 1994, AVIAT SPACE ENVIR MD, V65, P21
[9]  
Brenner M., 1985, Vocal Fold Physiology, Biomechanics, Acoustics, and Phonatory Control, P239
[10]   Interval estimation for a binomial proportion - Comment - Rejoinder [J].
Brown, LD ;
Cai, TT ;
DasGupta, A ;
Agresti, A ;
Coull, BA ;
Casella, G ;
Corcoran, C ;
Mehta, C ;
Ghosh, M ;
Santner, TJ ;
Brown, LD ;
Cai, TT ;
DasGupta, A .
STATISTICAL SCIENCE, 2001, 16 (02) :101-133