Model Compression for Data Compression: Neural Network Based Lossless Compressor Made Practical

In the recent few years, lossless compressors based on deep learning emerged as a new research direction. However, existing DL compressors focus mostly on compression ratio at the expense of compression speed (typically 10-1000 times slower than popular compressors, operating at the KB/s level), rendering them impractical to be deployed in industrial settings. To address the throughput issue of network inference, we propose to utilize model compression in DL-based lossless data compression. Our approach originates from the sparse learning framework to flexibly balance the model complexity and the compression ratio. Through systematic numerical experiments, we found that compressed models with only tens of thousands of parameters can still retain competitive compression ratios comparing with the original large models which are typically 100 times larger. The proposed scheme reveals novel properties concerning deep compression models and data-logarithmic relations between the model size and compressed data size for general mixed data, and a turning point indicating minimum representation size for relatively formatted data.