Real-time progressive compression method of massive data based on improved clustering algorithm

In order to realize the real-time progressive compression of massive data and ensure the quality of compressed data, a real-time progressive compression method of massive data based on improved clustering algorithm is proposed in this paper. Through the micro clustering stage of birch method based on K-Medoids clustering, Clustering Feature Tree hierarchy is constructed and numerical clustering features are extracted; Taking this feature as the input of macro clustering order, the Clustering Feature Tree leaf nodes are clustered based on the improved K-Medoids clustering method, and the clustering data cluster set is output; The set is used as the original data of real-time progressive compression, and the data is denoised and compressed by lifting format wavelet transform. On this basis, Huffman coding is used to compress the data losslessly. The test results show that this method has good clustering effect under the optimal number of clustering centers, can complete the real-time progressive compression of a large number of data, and the availability of compressed data is more than 92%.