Speaker count: a new building block for speaker diarization

In daily communication, several people sometimes talk simultaneously, resulting in overlapped speech segments. Such segments challenge machine listening tasks like speaker diarization or speech recognition. This paper presents a speaker diarization framework where speaker count, a building block to predict the number of active speakers in each analyzing audio window, is integrated. Such speaker count block can be developed independently with existing speaker diarization systems; its output is then used in the re-segmentation step of existing systems to better label active speakers in each considered window. We further investigate the effect of analyzing window size in diarization performance in an oracle setting. Our preliminary theoretical analysis shows that the overlap speech detection, a special case of speaker count, is helpful to reduce diarization error rate when the window size is small enough. Finally, experiment results obtained from two state-of-the-art diarization systems on a benchmark dataset confirm the potential benefit of the proposed approach.