Using Probabilistic Models for Data Compression

Abstract

Our research objective is to improve the Huffman coding efficiency by adjusting the data using a Poisson distribution, which avoids the undefined entropies too. The scientific value added by our paper consists in the fact of minimizing the average length of the code words, which is greater in the absence of applying the Poisson distribution. Huffman Coding is an error-free compression method, designed to remove the coding redundancy, by yielding the smallest number of code symbols per source symbol, which in practice can be represented by the intensity of an image or the output of a mapping operation. We shall use the images from the PASCAL Visual Object Classes (VOC) to evaluate our methods. In our work we use 10,102 randomly chosen images, such that half of them are for training, while the other half is for testing. The VOC data sets display significant variability regarding object size, orientation, pose, illumination, position and occlusion. The data sets are composed by 20 object classes, respectively: aeroplane, bicycle, bird, boat, bottle, bus, car, motorbike, train, sofa, table, chair, tv/monitor, potted plant, person, cat, cow, dog, horse and sheep. The descriptors of different objects can be compared to give a measurement of their similarity. Image similarity is an important concept in many applications. This paper is focused on the measure of similarity in the computer science domain, more specifically information retrieval and data mining. Our approach uses 64 descriptors for each image belonging to the training and test set, therefore the number of symbols is 64. The data of our information source are different from a finite memory source (Markov), where its output depends on a finite number of previous outputs. When dealing with large volumes of data, an effective approach to increase the Information Retrieval speed is based on using Neural Networks as an artificial intelligent technique.

On the Use of Variability Measures to Analyze Source Coding Data Based on the Shannon Entropy

“…Our work may be of particular interest to specialists in the field of cryptography in solving digital security problems [40]. Coding methods and their application in information security systems, cryptocompression systems, and data compression are relevant topics to be considered [10,41]. For example, the application of arithmetic coding methods in cryptographic information protection systems is considered in [42].…”

“…We are interested in identifying how pure each species is, and we assume that the source includes information variation when collecting the data. Because we do not know the true probability distribution of the source, we need to employ the plugin estimators defined in ( 9) and (10). We intuitively know that we have 100% knowledge of the Iris species because we have only a single species selected.…”

“…The concept of information theory and its relation to entropy is found in [7]. The reader is referred to [8][9][10] for applications of entropy in statistics to show the relevance of entropy. Two relevant works on statistical mechanics by maximizing entropy were presented in [11,12].…”

Holonomic and Non-Holonomic Geometric Models Associated to the Gibbs–Helmholtz Equation

“…Developments may include solving the analogue of Equations ( 15), (18), and (20), for other remarkable families of entropies (Renyi, Sharma-Taneja-Mittal, Naudts, etc). New examples are needed, in addition to the PDF solutions of normal type ( [71][72][73][74]).…”

Tsallis Entropy for Loss Models and Survival Models Involving Truncated and Censored Random Variables

“…Recent research results in statistics prove the increased interest for using different entropy measures. Many authors have dealt with this matter, among them are Koukoumis and Karagrigoriou [ 1 ], Iatan et al [ 2 ], Li et al [ 3 ], Miśkiewicz [ 4 ], Toma et. al.…”