Enhancing Dictionary Based Preprocessing for Better Text Compression

Abstract

With the rapid growing of data and number of applications, there is a crucial need of dictionary based reversible transformation techniques to increase the efficiency of the compression algorithms and hence contribute towards the enhancement in compression ratio. Performance analysis of compression methods in combination with the various transformation techniques is obtained for different text files of varying sizes. The popular block sorting lossless Burrows Wheeler Compression Algorithm (BWCA) is implemented along with one proposed method. For efficient compression a dictionary based transformation algorithm is also developed. It is observed that much increase in terms of compression ratio is attained when a source file is preprocessed with dictionary and then applied to BWCA and the proposed method.

Analisis Algoritma Mtf, MTF-1 Dan MTF-2 Pada Burrows Wheeler Compression Algorithm

“…Dalam modifikasinya, Baruah menambahkan satu tahap RLE antara BWT dan GST. Hasil yang didapatkan BWCA yang murni masih menghasilkan nilai rata-rata rasio kompresi yang lebih baik daripada yang dimodifikasi [2].…”

Experimental Analysis on the Performance of a New Preprocessing Method Used in Data Compression

“…In both the text transform methods [8,9], most frequently used words are listed at the beginning of the dictionary (upto 312 words). 2.…”

“…In both the methods after the first 312 words in the dictionary, the remaining words are stored in according to their lengths. Words with longer lengths are stored after words with shorter lengths in StarNT [8,9], while in our method the scenario is opposite.…”

“…In our method words having length ≤3, the code word is '#' followed by the word itself and words having length >3, the code words are assigned by different combinations of letters of English language But in StarNT [8,9], different combinations of letters of English language are only used as code words. 3.…”

“…In the encoded output text of our method, the '*' symbol is used to differentiate between the different code words. However, in StarNT [8,9] method, the '*' symbol is prefixed before a word in the encoded output file, when that word is not found in the dictionary.…”