Audio-visual biometric based speaker identification

In this paper, we present a multimodal audio-visual speaker identification system. The proposed system decomposes the information existing in a video stream into two components: speech and lip motion. It has been studied that lip information not only presents speech information but also characteristic information about a person's identity. Fusing this information with speech information will produce robust person identification tinder adverse condition. Gaussian mixture models (GMMs) and Hidden markov models (HMMs) are used throughout this work for the tasks of text dependent speaker recognition and month tracking. The performance is evaluated for dataset of 22 Indian of different ethnicity speakers each tittering a sentence. The results show that the performance of the biometric system is significantly better when both audio and video features are used.