High performance lossless compression of scientific floating data

被引：0
作者：
He K.-J. [1 ]
机构：
[1] School of Computer Science and Engineering, South China University of Technology
来源：
Jisuanji Xuebao/Chinese Journal of Computers | 2010年 / 33卷 / 06期
关键词：
Floating data; High performance; Lossless compression; Scientific computing;
D O I：
10.3724/SP.J.1016.2010.00966
中图分类号：
学科分类号：
摘要：
Scientific computing is playing more and more important role in the scientific research and in the industry, and enormous valuable scientific data are produced. Since the read/write speed of secondary storages (e.g. hard disks) is usually slow, besides occupying storage space, mass data also affect system performance. This paper studies the properties of floating-point data thoroughly, and establishes the theoretical relationship between the prediction precision and the compression ratio. By exploiting the interdependency between scientific data and making use of several predictor and high performance entropy encoding method, this paper proposes a high performance lossless compression method for scientific data. The method doesn't require users having priori knowledge about the data, nor is custom-designed predictor required. Comparing results with other methods, show the method has much higher compression ratio and constant high throughput. The method has been used for the compression of scientific data in large scale discrete element simulation of particle dynamics.
引用
收藏
页码：966 / 976
页数：10
相关论文
共 21 条
  • [11] IEEE 754-2008: Standard for floating-point arithmetic, (2008)
  • [12] Mahoney M., Adaptive Weighing of Context Models for Loss-Less Data Compression, (2005)
  • [13] Cleary J.G., Witten I.H., Data compression using adaptive coding and partial string matching, IEEE Transactions on Communications, 32, 4, pp. 396-402, (1984)
  • [14] Sazeides Y., Smith J.E., The predictability of data values, Proceedings of the 30th International Symposium on Microarchitecture, pp. 248-258, (1997)
  • [15] Goeman B., Vandierendonck H., Bosschere K., Differential FCM: Increasing value prediction accuracy by improving table usage efficiency, Proceedings of the International Symposium on High Performance Computer Architecture, pp. 207-216, (2001)
  • [16] Martin G.N.N., Range encoding: An algorithm for removing redundancy from a digitized message, Video and Data Recording Conference, (1979)
  • [17] Schindler M., A fast renormalization for arithmetic coding, Proceedings of the 1998 Data Compression Conference, (1998)
  • [18] He K.J., Dong S.B., Zhou Z.Y., Multigrid contact detection method, Physical Review E, 75, 3, (2007)
  • [19] Zhong W.Z., He K.J., Zhou Z.Y., Xia W., Li Y.Y., Calibration of the damping coefficient for the DEM simulation, Acta Physica Sinica, 58, 8, pp. 5155-5161, (2009)
  • [20] Zhong W.Z., He K.J., Zhou Z.Y., Xia W., Li Y.Y., Physical model and simulation system of powder packing, Acta Physica Sinica, 58, 13, (2009)
123