Search engine intelligent algorithm for big data

How to improve the efficiency of search engine in big data environment has become a hot issue. In this study, for clustering search, key words were extracted by Term Frequency-Inverse Document Frequency (TF-IDF), the defect of K-means algorithm was improved by combining Canopy algorithm to obtain a Canopy-K-means (CKM) algorithm, and its retrieval performance was tested. The results showed that the performance of the algorithm increased with the increase of data volume in searching different key words, the search time shortened, and the recall factor and the pertinency factor improved. The CKM algorithm showed an excellent performance in big data processing and better performance compared to LDA and K-means algorithms. The comparison with the clustering performance of K-means algorithm demonstrated that the clustering result of the CKM algorithm was more similar to the actual number of clusters and its clustering accuracy was higher, indicating that the CKM algorithm was effective in retrieval. The experimental results of this study make some contributions to improve the efficiency of data retrieval and meet the needs of users, which is conducive to the better development of search engines. © 2020 Begell House Inc.. All rights reserved.