Using Probabilistic Models for Data Compression

被引:4
|
作者
Iatan, Iuliana [1 ]
Dragan, Mihaita [2 ]
Dedu, Silvia [3 ]
Preda, Vasile [2 ,4 ,5 ]
机构
[1] Tech Univ Civil Engn, Dept Math & Comp Sci, Bucharest 020396, Romania
[2] Univ Bucharest, Fac Math & Comp Sci, Bucharest 010014, Romania
[3] Bucharest Univ Econ Studies, Dept Appl Math, Bucharest 010734, Romania
[4] Gheorghe Mihoc Caius Iacob Inst Math Stat & Appl, Bucharest 050711, Romania
[5] Costin C Kiritescu Natl Inst Econ Res, Bucharest 050711, Romania
关键词
data compression; descriptors; probabilistic models; entropy; Huffman coding; coding redundancy; coding efficiency; artificial intelligence; ENTROPY; PATTERN;
D O I
10.3390/math10203847
中图分类号
O1 [数学];
学科分类号
0701 ; 070101 ;
摘要
Our research objective is to improve the Huffman coding efficiency by adjusting the data using a Poisson distribution, which avoids the undefined entropies too. The scientific value added by our paper consists in the fact of minimizing the average length of the code words, which is greater in the absence of applying the Poisson distribution. Huffman Coding is an error-free compression method, designed to remove the coding redundancy, by yielding the smallest number of code symbols per source symbol, which in practice can be represented by the intensity of an image or the output of a mapping operation. We shall use the images from the PASCAL Visual Object Classes (VOC) to evaluate our methods. In our work we use 10,102 randomly chosen images, such that half of them are for training, while the other half is for testing. The VOC data sets display significant variability regarding object size, orientation, pose, illumination, position and occlusion. The data sets are composed by 20 object classes, respectively: aeroplane, bicycle, bird, boat, bottle, bus, car, motorbike, train, sofa, table, chair, tv/monitor, potted plant, person, cat, cow, dog, horse and sheep. The descriptors of different objects can be compared to give a measurement of their similarity. Image similarity is an important concept in many applications. This paper is focused on the measure of similarity in the computer science domain, more specifically information retrieval and data mining. Our approach uses 64 descriptors for each image belonging to the training and test set, therefore the number of symbols is 64. The data of our information source are different from a finite memory source (Markov), where its output depends on a finite number of previous outputs. When dealing with large volumes of data, an effective approach to increase the Information Retrieval speed is based on using Neural Networks as an artificial intelligent technique.
引用
收藏
页数:29
相关论文
共 50 条
  • [41] Fuzzy data compression for energy optimization models
    Groscurth, HM
    Kress, KP
    ENERGY, 1998, 23 (01) : 1 - 9
  • [42] Advances in the data compression of digital elevation models
    Kidner, DB
    Smith, DH
    COMPUTERS & GEOSCIENCES, 2003, 29 (08) : 985 - 1002
  • [43] Data compression using antidictionaries
    Crochemore, M
    Mignosi, F
    Restivo, A
    Salemi, S
    PROCEEDINGS OF THE IEEE, 2000, 88 (11) : 1756 - 1768
  • [44] Probabilistic Models to Reconcile Complex Data from Inaccurate Data Sources
    Blanco, Lorenzo
    Crescenzi, Valter
    Merialdo, Paolo
    Papotti, Paolo
    ADVANCED INFORMATION SYSTEMS ENGINEERING, PROCEEDINGS, 2010, 6051 : 83 - 97
  • [45] Probabilistic Power Flow Using Novel Wind and Solar Probabilistic Models
    Nassar, M. E.
    Salama, M. M. A.
    2016 IEEE POWER AND ENERGY SOCIETY GENERAL MEETING (PESGM), 2016,
  • [46] Reconstruction of sequential data with probabilistic models and continuity constraints
    Carreira-Perpiñán, MA
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 12, 2000, 12 : 414 - 420
  • [47] Optimal regularizations for data generation with probabilistic graphical models
    Fanthomme, A.
    Rizzato, F.
    Cocco, S.
    Monasson, R.
    JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2022, 2022 (05):
  • [48] Probabilistic Topic Models for Text Data Retrieval and Analysis
    Zhai, ChengXiang
    SIGIR'17: PROCEEDINGS OF THE 40TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, 2017, : 1399 - 1401
  • [49] ToPS: A Framework to Manipulate Probabilistic Models of Sequence Data
    Kashiwabara, Andre Yoshiaki
    Bonadio, Igor
    Onuchic, Vitor
    Amado, Felipe
    Mathias, Rafael
    Durham, Alan Mitchell
    PLOS COMPUTATIONAL BIOLOGY, 2013, 9 (10)
  • [50] A probabilistic approach to integrating dynamic data in reservoir models
    Kashib, T
    Shnivasan, S
    JOURNAL OF PETROLEUM SCIENCE AND ENGINEERING, 2006, 50 (3-4) : 241 - 257