String Matching Over Compressed Text on Handheld Devices Using Tagged Sub-Optimal Code (TSC)

Abstract

This paper presents Tagged Sub-optimal code (TSC), a new coding technique to speed up string matching over compressed databases on personal digital assistants (PDA). TSC is a variable-length sub-optimal code that supports minimal prefix property. It always determines its codeword boundary without traversing a tree or lookup table. TSC technique may be beneficial in many types of applications: speeding up string matching over compressed text, and speeding decoding process. This paper also presents two algorithms for string matching over compressed text using TSC (SCTT) and the Byte Pair Encoding (BPE) technique (SCTB).Several experiments were conducted to compare the performance of TSC, Byte Pair Encoding (BPE), and Huffman code. Several PDA databases with different record sizes were used: the well-known Calgary dataset and a set of small-sized PDA databases. Experimental results show that SCTT is almost twice as fast as the Huffmanbased algorithm. SCTT has also the same performance in search time as the search in uncompressed databases and is faster than the SCTB algorithm. For frequently updated PDA databases such as phone books, to-do list, and memos, SCTT is the recommended method regardless of the size of the average record length, since the time required to compress the updated records using BPE poses significant delays compared to TSC.

Speeding up string matching over compressed text on handheld devices using tagged sub-optimal code (TSC)

“…A solution to compress the entire database and keeps it accessible and manageable while it is on compressed form is introduced in our previous work [29], which is done by compressing each record individually and retain the record references structure within the database unbroken.…”

“…The approach of compression we used in this work is slightly different than our previous work [29] in which the entire database will be compressed as one intact file and then store each individual compressed record in its physical location within the database structure to allow random access to individual records. Figure 1 shows the logical representation of the proposed compression solution to the entire database.…”

“…In our previous work [29] Byte Pair Encoding (BPE) has been chosen as the compression algorithm for handheld databases since it works efficiently and effectively in small text in size and supports partial decompression. However, BPE has significant delay in the compression process and the look up tables that are built during the compression process have to be stored in each individual record, which slow the search process.…”

“…The implementation of the new algorithm for searching over compressed text by using BPE is presented in our previous work [29].…”

“…Consequently, string matching technique over compressed text on handheld devices is required. In our previous work [29], we developed an algorithm for searching over compressed text on handheld devices by using BPE scheme, which works very fast and achieved 6.6 times faster than decompressing the databases followed by simple search. On the other hand, BPE scheme involves significant delay in the compression process.…”