This paper introduces the concept of outlier detection into blind identification of underdetermined mixtures. We propose a mixing matrix estimation algorithm based on outlier detection. First calculate the spatial Time-Frequency (TF) distribution of the mixtures, detect the single source points in the TF domain, and then detect the outliers, remove them from the set of single source points, and finally estimate the mixing matrix using a clustering method. The proposed algorithm relaxes the condition on the sparsity of sources. The mixing matrix estimation accuracy is improved by detecting the outliers and removing them, which is also helpful for the estimation of the number of sources. Simulation results show that the proposed algorithm estimates the mixing matrix with high accuracy and robustness compared with other algorithms.