Optimizing Timestamp Management in Data Stream Management Systems

Logical Foundations of Continuous Query Languages for Data Streams

“…A much studied DSMS problem is how to best ensure that binary query operators, such as unions or joins, generate outputs sorted by increasing timestamp values [14,5,6]. To derive a logic-based characterization of this problem, assume that our msg stream is in fact built by combining the two message streams sensr1 and sensr2.…”

“…Thus, for the union in Figure 1, when one of the two input buffers is empty, we cannot take the first item from the other buffer, until we know what its timestamp value will be. This problem has been extensively studied, but only at the implementation level [14,5,6]. At the logical level the problem can be solved as follows:…”

“…As a result, this operator might have to enter an idle-waiting state that is akin to temporary blocking [5].…”

“…For instance, for the query at hand, the DSMS system might in fact enforces conditions such as ¬missing2(T1) in the first rule of our union. Efficient support of these PSA-emulating conditions requires the use of sophisticated techniques, such as intelligent backtracking [5]. For instance in Figure 1, say that the lower buffer feeding the union has a tuple with timestamp t 1 , while the other buffer is empty.…”

“…Finally, if the buffer is empty, an additional backtracking step will be performed to visit the buffer supplied by Source1, and so on. As discussed in [5], this backtracking approach can lead to significant improvements in the response time of our DSMS. Although space limitations prevent us from discussing this approach further, we observe that (i) while Streamlog is clearly inspired by Datalog, its execution exploits Prolog's backtracking mechanism, and (ii) the techniques used to support PSA are also very useful to control and expedite execution of single-stream queries.…”