On the memory requirements of XPath evaluation over XML streams

Abstract

The important challenge of evaluating XPath queries over XML streams has sparked much interest in the past few years. A number of algorithms have been proposed, supporting wider fragments of the query language, and exhibiting better performance and memory utilization. Nevertheless, all the algorithms known to date use a prohibitively large amount of memory for certain types of queries. A natural question then is whether this memory bottleneck is inherent or just an artifact of the proposed algorithms.In this paper we initiate the first systematic and theoretical study of lower bounds on the amount of memory required to evaluate XPath queries over XML streams. We present a general lower bound technique, which given a query, specifies the minimum amount of memory that any algorithm evaluating the query on a stream would need to incur. The lower bounds are stated in terms of new graph-theoretic properties of queries. The proofs are based on tools from communication complexity.We then exploit insights learned from the lower bounds to obtain a new algorithm for XPath evaluation on streams. The algorithm uses space close to the optimum. Our algorithm deviates from the standard paradigm of using automata or transducers, thereby avoiding the need to store large transition tables.

Efficient algorithms for descendant-only tree pattern queries

“…It follows that the mapping y also matches the path hparentðq 0 Þ Á Á Á root ðQ Þi as high as possible into the sub-path hd 0 Á Á Á root ðDÞi. As d 0 is an ancestor of d þ 1 (line 21) we now have that the mapping y matches the path hparentðq 0 Þ Á Á Á root ðQ Þi as high as possible into the path hparentðd þ 1Þ Á Á Á root ðDÞi, which proves (2). In order to prove (3), i.e., to prove that the path hparentðd þ 1Þ Á Á Á root ðDÞi cannot match the path hq 0 Á Á Á root ðQ Þi, we consider two cases: If q ¼ q 0 , by the induction hypothesis, the path hparentðdÞ Á Á Á root ðDÞi cannot match the path hq Á Á Á root ðQ Þi.…”

“…To this end, we first store d (resp., q) in a temporary variable v. We continue following the parent relation in this fashion until we find u, at which point we return the value of v, which is a child of u. 2 In more detail, for given input nodes d and q the LOGSPACE procedure tests whether d matches q and based on the result of this test it computes the next function call. This is a rather extensive case study.…”

“…Certain algorithms aim at processing the data tree as a stream (i.e., in a single scan) [2,3,5,6,9,14,[16][17][18]. For this case a number of lower bound results have been obtained using mechanisms from communication complexity [2,3,15]. It is basically known that streaming algorithms for even simple tree patterns consume space proportional to the size of the data tree in the worst case.…”

Evaluation Techniques for Generalized Path Pattern Queries on XML Data

“…Given a path query Q and an XML tree T, let input denote the sum of sizes of the input streams, output denote the size of the answer of Q on T, and |Q| denote the number of nodes in Q. The recursion depth of a query node X in T is the maximum number of nodes in a path of T that match X [3]. We define the recursion depth of Q in T, denoted recur Depth, as the maximum of the recursion depths of the query nodes of Q in T. Clearly, recur Depth is bounded by the maximum number of occurrences of a query label in a path of T. We assume that query stacks fit in memory and all stack operations are conducted in memory Clearly, assuming that the size of the query is insignificant compared to the size of data, PathStack-R is asymptotically optimal for path queries with repeated labels.…”

Worst-case optimal algorithm for XPath evaluation over XML streams

“…Bar-Yossef et al [5] presented an algorithm that uses O (r|Q |(log |Q | + log d)) bits of space and O (r|Q ||D|) time. They also presented an Ω(r + log d) space lower bound, for each instance (Q , D).…”