Current data integration approaches are moving towards real-time updates. One important element in real-time data integration is the join of a continuous incoming data stream with a disk-resident relation. Because data stream is infinite, it is impossible to adopt blocking join algorithms such as sort-merge and hash join. The novel algorithm MESHJOIN has been proposed for joining a continuous stream with a disk-resident relation. The crux of MESHJOIN algorithm is that the whole memory block of disk-based relation is replaced at each iteration. We propose that the memory block is divided into a number of logical partitions, and then only one logical partition of memory block is replaced at each iteration. The experimental results show that the service rate of the join is increased because I/O cost for one loop iteration is decreased. Keywords-relational data stream; join; disk-resident relation; logical partition I.0 B INTRODUCTION Active data warehousing has emerged as a new business intelligence (BI) paradigm where updates from the operational stores are propagated in real-time to the repository. This shift of practices significantly affects the ETL (extract transfer load) process as the type of joins between an infinite stream of incoming records and some stored data warehouse table. The core operation is identified byR where S is viewed as append-only sequences of data tuples where all tuples of the same stream have the same schema, R is a large, disk-resident, warehouse relation, and the join condition C depends on the semantics of the transformation. An active data warehouse requires the evaluation of this expression on line as the tuples of S are streamed from the operational sources, in order to ensure that the updates are propagated in a timely fashion. The major challenge is to handle the fast arrival rate of S tuples relative to the slow I/O access of R.While there is a lot of research on how to join relations that are either both stored or streaming, there is little work devoted to the problem of joining a streaming relation with a store one. Whether we can adapt blocking join algorithms such as sort-merge [15] and hash join to the join C S R , it will require a considerable amount of disk buffering for S tuples which would slow down the join operation [13,14]. The indexed nested-loop join also satisfy the setting, but it may require the maintenance of an additional index on R and probe the index incurs expensive random I/Os. The work in [1] first drew attention to the need to support streaming updates in an active data warehouse. The proposed MESHJOIN algorithm utilizes sequential reads from the stored relation in a round robin fashion in order to amortize the I/O look up cost for a large number of stream tuples. The work in [3,4,5] proposes some revised algorithm.In this paper we contribute by introducing block stream algorithm for joining a relational stream with a disk-resident relation. We increase the stream buffer so that memory can accommodate more windows, thereby reducing disk block I/O. In t...

Sliding windows of data stream have rich semantics, which results all kinds of window semantics of different data stream, so join semantics between the different types of windows becomes very complicated. The basic join semantic of data streams, the join semantic of tuple-based sliding window and the join semantic of time-based sliding window have partly solved the semantics of stream joins, but the heterogeneity of sliding windows is difficult to be solved. In this paper we present the join semantic model based on matching window identifies for joining of multi-data stream. We make use of window identifies to shield the difference of window attribute, window size, and window slide. In this paper, a sliding window is divided into a number of sub-windows when the newest sub-window fills up it and it is appended to the sliding window while the oldest sub-window in the sliding window is removed. We use the equivalence relation of overlapping sub-window belonging to the adjacent sliding window to reduce the number of join computing. We propose the corresponding algorithm of window join to maintain the window. The theoretical and experimental analysis show that the joining model of window identifies can synchronize multiple data stream.

No abstract

No abstract