Semantics and evaluation techniques for window aggregates in data streams

Abstract

A windowed query operator breaks a data stream into possibly overlapping subsets of data and computes a result over each. Many stream systems can evaluate window aggregate queries. However, current stream systems suffer from a lack of an explicit definition of window semantics. As a result, their implementations unnecessarily confuse window definition with physical stream properties. This confusion complicates the stream system, and even worse, can hurt performance both in terms of memory usage and execution time. To address this problem, we propose a framework for defining window semantics, which can be used to express almost all types of windows of which we are aware, and which is easily extensible to other types of windows that may occur in the future. Based on this definition, we explore a one-pass query evaluation strategy, the Window-ID (WID) approach, for various types of window aggregate queries. WID significantly reduces both required memory space and execution time for a large class of window definitions. In addition, WID can leverage punctuations to gracefully handle disorder. Our experimental study shows that WID has better execution-time performance than existing window aggregate query evaluation options that retain and reprocess tuples, and has better latencyaccuracy tradeoffs for disordered input streams compared to using a fixed delay for handling disorder.

Design and Evaluation of a Configurable Query Processing Hardware for Data Streams

“…Query Q 1 introduces three parameters: RANGE, SLIDE, and WATTR. RANGE indicates the size of the windows; SLIDE indicates the step by which the windows move; WATTR indicates the windowing attribute-the attribute over which RANGE and SLIDE are specified [24]. Given the specification above, a bid stream is divided into overlapping 4-minute windows starting every minute.…”

Temporal Coalescing on Window Extents over Data Streams

“…Simply speaking, it only means substituting a window extent for a temporal table. Unlike database queries, however, stream tuples arrive unboundedly and stream queries run continuously, typically on an ever-changing subset of the stream referred to as a window extent [16]. This makes both coalescing schemes disable the basic window extent update algorithms.…”

“…A window may be tuple-based or timebased [17], [18]. At any time instant t, a tuple-based window of size w (tuples) on a data stream contains tuples with the largest w timestamps not exceeding t and a time-based window of size w (e.g., seconds) contains tuples with the timestamps in the range of t − w to t. The set of physical tuples contained in a window is referred to as a window extent, and the specification of a window extent is done through the window operator [16]. In other words, a window operator is like a "cookie cutter" and window extents are like "cookies cut" with it [16].…”

Window Update Patterns in Stream Operators

“…It is exactly this finite portion of stream contents that can be considered at each evaluation; such an operation is abstracted through a windowing construct by setting specific constraints over time, number of tuples or other stream properties. A windowing attribute [14] is necessary for establishing order among stream items, and timestamps serve this purpose perfectly.…”

“…Several optimization techniques were applied for reducing the overhead of doubling the amount of tuples processed and for avoiding output delays. Besides, a detailed examination of stream aggregates was proposed in [14]; under this interpretation, windowed aggregation reduces to a simple relational one. A temporal stream algebra [13] covers sliding and fixed windows only, distinguishing logical and physical operator levels for query specification and evaluation.…”