Binaural Speech Enhancement based on DNN for the Application of Virtual Reality

Binaural sound can increase the immersion in virtual reality scenes due to the sense of direction, but when recorded in real-world, it may be corrupted by noise. Some of the existing binaural speech enhancement or separation methods can only provide the single-channel output, which will lead to the loss of the sense of direction. Some methods can provide the dual-channel output, however, such methods will suffer performance loss when the binaural clean speeches and the binaural noise are in the same direction. In this paper, we propose a binaural speech enhancement method based on deep neural network, aiming at dealing with the situation that binaural clean speeches and binaural noises are in the same direction. By mapping the features of the binaural noisy speeches to the labels of the binaural clean speeches, the dual-channel output can be obtained. Besides, batch normalization layer is introduced to further improve the performance. Compared with the baseline methods, the proposed method can obtain better speech quality and intelligibility, and the sense of the direction of the estimated binaural speeches can also be better preserved.