Fast and Lightweight LCP-Array Construction Algorithms

Abstract-To spot malicious manipulation, remote attestation and maintenance for devices that are under legal control is very important. One example are measuring instruments, where the manufacturer and the market surveillance want to check if system integrity is preserved. In Europe, legal requirements state that a software identifier needs to be supplied/output by the device, which is often just a checksum over the files that are considered to be legally relevant for the measuring purpose. As measuring instruments and also other legally monitored devices are often small embedded systems, the need for a fast algorithm arises that creates a small file system list containing as much information as possible. In this paper, a new file system structure called FLOUDS is explained that fulfills these requirements. The FLOUDS uses theoretical optimal space to represent the file system structure, while it, nevertheless, enables fast file searches by names and also properties. For example, all files of a specific file type, e.g., pictures, movies, executables, etc., can be listed in O(p lg n) time, where p is the number of files of the specific file type searched for, and, where n represents the total number of file types in the system.

show abstract

“…In nearly all cases, attempts were made at practical implementations, with successful results [20], [24], [31].…”

Section: Succinct Data Structuresmentioning

confidence: 99%

FLOUDS: A Succinct File System Structure

Peters

Fischer

Thiel

et al. 2017

Annals of Computer Science and Information Systems

View full text Add to dashboard Cite

show abstract

“…This approach seems even less suitable for an efficient implementation. There are only a couple of semi-external construction algorithms [15,18,33], where "semi-external" means that they only need some arrays in RAM, while other parts can be scanned.…”

Section: Introductionmentioning

confidence: 99%

Inducing Suffix and LCP Arrays in External Memory

Bingmann

Fischer

Osipov

2016

ACM J. Exp. Algorithmics

View full text Add to dashboard Cite

We consider text index construction in external memory (EM). Our first contribution is an inducing algorithm for suffix arrays in external memory. Practical tests show that this outperforms the previous best EM suffix sorter [Dementiev et al., ALENEX 2005] by a factor of about two in time and I/O-volume. Our second contribution is to augment the first algorithm to also construct the array of longest common prefixes (LCPs). This yields the first EM construction algorithm for LCP arrays. The overhead in time and I/O volume for this extended algorithm over plain suffix array construction is roughly two. Our algorithms scale far beyond problem sizes previously considered in the literature (text size of 80 GiB using only 4 GiB of RAM in our experiments).

show abstract

“…The first LACA by Kasai et al [14] is simple and runs in linear time but needs a lot a space (the text plus 3n integers). Several later algorithms aimed at reducing the space [15,17,22,11,7,2]. Some of the algorithms can even be made semi-external, i.e., they keep most of the data structures on disk but need to have at least the full text in RAM [22,11].…”

Section: Introductionmentioning

confidence: 99%

LCP Array Construction in External Memory

Kärkkäinen

Kempa

2014

Experimental Algorithms

View full text Add to dashboard Cite

Abstract.One of the most important data structures for string processing, the suffix array, needs to be augmented with the longest-commonprefix (LCP) array in numerous applications. We describe the first external memory algorithm for constructing the LCP array given the suffix array as input. The only previous way to compute the LCP array for data that is bigger than the RAM is to use a suffix array construction algorithm with complex modifications to produce the LCP array as a by-product. Compared to the best prior method, our algorithm needs much less disk space (by more than a factor of three) and is significantly faster. Furthermore, our algorithm can be combined with any suffix array construction algorithm including a better one developed in the future.

show abstract

Fast and Lightweight LCP-Array Construction Algorithms

Cited by 48 publications

References 10 publications

FLOUDS: A Succinct File System Structure

FLOUDS: A Succinct File System Structure

Inducing Suffix and LCP Arrays in External Memory

LCP Array Construction in External Memory

Contact Info

Product

Resources

About