Assessment of Virtual Reality Motion Sickness Severity Based on EEG via LSTM/BiLSTM

Virtual reality motion sickness (VRMS), which is mainly caused by visual-vestibular-somatic conflict, is a risk to the health and security of users who have fun in virtual reality (VR). To evaluate the VRMS in VR experience, this article introduces regression models based on recurrent neural network to predict the user's VRMS severity from their EEG data. The EEG before, during, and after a VRMS exposure task is collected and divided into five rhythms as the inputs. The simulator sickness questionnaire (SSQ) is performed after each task to label the EEG data as the VRMS severity. In the proposed four regression models, four different approaches used to extract electrode-frequency features are combined with a three-layer long short-term memory (LSTM) or bi-directional long short-term memory (BiLSTM) network used to study the temporal features. Besides, the different numbers of hidden units in LSTM and BiLSTM network are compared to choose an optimal one. The results suggest the fusion of all electrodes and rhythms information using convolutional neural network before inputting the LSTM/BiLSTM network provides the best regression performance, in which the mean MSE and R2-score are 53.6159 and 0.8683, respectively. The work introduced in this article provides a method to assess the performance of VR productions and is an objective and direct guideline to overcome VRMS and optimize VR systems.