Engineering succinct DOM

Abstract

We describe the engineering of Succinct DOM (SDOM), a DOM implementation, written in C++, which is suitable for in-memory representation of large static XML documents. SDOM avoids the use of pointers, and is based upon succinct data structures, which use an information-theoretically minimum amount of space to represent an object.SDOM gives a space-efficient in-memory representation, with stable and predictable memory usage. The space used by SDOM is an order of magnitude less than that used by a standard C++ DOM representation such as Xerces, but SDOM is extremely fast: navigation is in some cases faster than for a pointer-based representation such as Xerces (even for moderate-sized documents which can comfortably be loaded into main memory by Xerces).A variant, SDOM-CT, applies bzip-based compression to textual and attribute data, and its space usage is comparable with "queryable" XML compressors. Some of these compressors support navigation and/or querying (e.g. subpath queries) of the compressed file. SDOM-CT does not support querying directly, but remains extremely fast: it is several orders of magnitude faster for navigation than queryable XML compressors that support navigation (and only a few times slower than say Xerces).

Dynamizing Succinct Tree Representations

“…Although a number of real-life "benchmark" XML files are available [9], we did not use them extensively, for a variety of reasons. Firstly, the files were relatively small -the largest, although 600MB in size, had only about 25 million nodes: if stored in the information-theoretic minimum amount of space, the tree would almost completely fit into a 6MB cache.…”

“…We considered two kinds of traversals: a non-recursive depth-first traversal (DFS) and the all root-leaf (ALR) traversal [9], where we do DFS, but whenever the DFS encounters a leaf we trace the path back to the root. Input trees and traversals.…”

“…For example, when representing XML documents, which are essentially ordinal trees, a standard pointer-based representation [1] has five pointers per node 1 (320 bits per node on a 64-bit machine) to represent the tree structure and support fast navigation; thus, the attractiveness of a representation that takes just a few bits per node but supports operations quickly in practice, is clear. Indeed, succinct ordinal trees have been successfully applied to several XML applications [25,9,3,7].…”

2LP: A double-lazy XML parser

“…Some others, considered query-friendly [46], only require a small subset of the document to be de-compressed. Some recent works [47][48][49] can support navigation in the compressed document. SDOM [47] proposes a succinct way of representing XML documents in order to reduce their memory fingerprint and allow efficient navigation.…”

Succinct Representations of Ordinal Trees

“…They have also only been integrated successfully into a range of applications [13,3,45]. Furthermore, they are currently available in well-documented open-source libraries such as SDSL [22] and succinct [37].…”