An Improved K-means Algorithm for Document Clustering

K-Means algorithm has a major shortcoming of high dimensional and sparse data. So the traditional measurement of the distance can't deal with the data effectively. Motivated by this, this paper proposed a K-Means algorithm based on SimHash. After preprocessing of the text, SimHash is used to calculate the feature vectors extracted, and then the fingerprint of each text is obtained. SimHash not only reduces the dimension of the text, but also directly calculates the Hamming distance between the fingerprints as the vector distance. According to the Hamming distance, it can judge which clustering the data is belongs to. Experimental result shows that the algorithm guarantees the quality of the clustering, and greatly reduces the speed of K-means clustering algorithm.