A Disk-Based, Adaptive Approach to Memory-Limited Computation of Windowed Stream Joins

Chakraborty, Abhirup; Singh, Ajit

doi:10.1007/978-3-642-15364-8_19

Cited by 5 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Adaptive, Hash-partitioned Exact Window Join (AH-EWJ) [22] has been introduced to produce accurate results for sliding window joins over data streams. AH-EWJ can also be used to join a stream with disk-based data.…”

Section: Related Workmentioning

confidence: 99%

Skewed distributions in semi-stream joins: How much can caching help?

Naeem

Dobbie

Lutteroth

et al. 2017

Information Systems

View full text Add to dashboard Cite

Semi-stream join algorithms join a fast data stream with a disk-based relation. This is important, for example, in real-time data warehousing where a stream of transactions is joined with master data before loading it into a data warehouse. In many important scenarios, the stream input has a skewed distribution, which makes certain performance optimisations possible.We propose two such optimisation techniques: 1) a caching technique for frequently used master data, and 2) a technique for selective load shedding of stream tuples. The caching technique is fine-grained, operating on a tuple-level. Furthermore, it is generic in the sense that it can be applied to different semistream join algorithms to deal with high loads. We analyze it by combining it with various well-known semi-stream joins, and show that it improves the service rate by more than 40% for typical data with skewed distributions. The load shedding technique sheds the fraction of the stream that is most expensive to join. In contrast to existing approaches, the service rate improves under load shedding. We present experimental data showing significant improvements as compared to related approaches and perform a sensitivity analysis for various internal parameters.

show abstract

Section: Related Workmentioning

confidence: 99%

Skewed distributions in semi-stream joins: How much can caching help?

Naeem

Dobbie

Lutteroth

et al. 2017

Information Systems

View full text Add to dashboard Cite

show abstract

“…The join result of the data stream that do not use windows can be seen as the join view [14] of the append-only package. And the join process of data stream S1 and S2 can be seen as the join view of append-only package S1 and S2 which updates and maintains the package view when the package S1 or S2 is updating.…”

Section: The Basic Model Of the Join Of Data Streammentioning

confidence: 99%

Research on Sliding Window Join Semantics and Join Algorithm in Heterogeneous Data Streams

Wei¹,

Xianxia²

2015

TOCSJ

View full text Add to dashboard Cite

Sliding windows of data stream have rich semantics, which results all kinds of window semantics of different data stream, so join semantics between the different types of windows becomes very complicated. The basic join semantic of data streams, the join semantic of tuple-based sliding window and the join semantic of time-based sliding window have partly solved the semantics of stream joins, but the heterogeneity of sliding windows is difficult to be solved. In this paper we present the join semantic model based on matching window identifies for joining of multi-data stream. We make use of window identifies to shield the difference of window attribute, window size, and window slide. In this paper, a sliding window is divided into a number of sub-windows when the newest sub-window fills up it and it is appended to the sliding window while the oldest sub-window in the sliding window is removed. We use the equivalence relation of overlapping sub-window belonging to the adjacent sliding window to reduce the number of join computing. We propose the corresponding algorithm of window join to maintain the window. The theoretical and experimental analysis show that the joining model of window identifies can synchronize multiple data stream.

show abstract

“…• OpenCQ [8] [12,13,14] 임의로 표본을 추출하는 것 대신에 최근 스트림 데이터를 주기적으로 갱신하여 저장 관리 • 히스토그램, 웨이블릿 [15,16,17,18] 스트림 데이터의 빈도 분포를 이용한 대략적인 요약 정보 저장 관리 마이닝 기술…”

unclassified

A Sensor Data Management System for USN based Fire Detection Application

Park¹,

Kim²

2011

Journal of the Korea Society of Computer and Information

View full text Add to dashboard Cite

These days, the research of a sensor data management system for USN based real-time monitoring application is active thanks to the development and diffusion of sensor technology. The sensor data is rapidly changeable, continuous and massive row level data. However, end user is only interested in high level data. So, it is essential to effectively process the row level data which is changeable, continuous and massive. In this paper, we propose a sensor data management system with multi-analytical query function using OLAP and anomaly detection function using learning based classifier. In the experimental section, we show that our system is valid through

show abstract

A Disk-Based, Adaptive Approach to Memory-Limited Computation of Windowed Stream Joins

Cited by 5 publications

References 18 publications

Skewed distributions in semi-stream joins: How much can caching help?

Skewed distributions in semi-stream joins: How much can caching help?

Research on Sliding Window Join Semantics and Join Algorithm in Heterogeneous Data Streams

A Sensor Data Management System for USN based Fire Detection Application

Contact Info

Product

Resources

About