Automatic fast double KNN classification algorithm based on ACC and hierarchical clustering for big data

In data mining, k-nearest neighbors (KNN) classifier is an efficient lazy learning yet simple widely renowned method, which has been widely used in many actual applications, successfully. Because of time and memory restrictions of KNN, when KNN is tested in large-scale datasets, the classification accuracy is very low. Therefore, we propose an automatic fast double KNN classification algorithm on the basis of automatically determining the cluster centers and hierarchical clustering. We introduce automatically determining the cluster centers into the KNN in training process. Namely, big data samples are divided into several parts depending on our clustering methods. Afterwards, the clusters nearest to testing samples are excavated as the new training samples in the testing process. Each of the new samples is then conducted with hierarchical clustering. In this way, computation and time complexity are greatly reduced. Finally, experiments results conducted on big data show that new KNN classification method can significantly raise the accuracy and efficiency of automatic classification than other state-of-the-art KNN classification algorithms.