SemCast: Semantic Multicast for Content-Based Data Dissemination

Papaemmanouil, Olga; Çetintemel, Uǧur

doi:10.1109/icde.2005.131

Cited by 47 publications

(44 citation statements)

References 24 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They assume that there is an efficient scheme to organize the distributed dissemination servers and employ the schemes of general multicast systems. More recently, authors in [16] proposed a scheme to assign the dissemination servers to different dissemination channels based on the containment relationships of the user profiles. However this paper also does not focus on dissemination tree construction algorithms and does not address the coherenecy problems.…”

Section: Related Workmentioning

confidence: 99%

Disseminating streaming data in a dynamic environment: an adaptive and cost-based approach

2007

View full text Add to dashboard Cite

In a distributed stream processing system, streaming data are continuously disseminated from the sources to the distributed processing servers. To enhance the dissemination efficiency, these servers are typically organized into one or more dissemination trees. In this paper, we focus on the problem of constructing dissemination trees to minimize the average loss of fidelity of the system. We observe that existing heuristic-based approaches can only explore a limited solution space and hence may lead to sub-optimal solutions. On the contrary, we propose an adaptive and cost-based approach. Our cost model takes into account both the processing cost and the communication cost. Furthermore, as a distributed stream processing system is vulnerable to inaccurate statistics, runtime fluctuations of data characteristics, server workloads, and network conditions, we have designed our scheme to be adaptive to these situations: an operational dissemination tree may be incrementally transformed to a more cost-effective one. Our adaptive strategy employs distributed decisions made by the distributed servers independently based on localized statistics collected by each server at runtime. For a relatively static environment, we also propose two static tree construction algorithms relying on apriori system statistics. These static trees can also be used as initial trees in a dynamic environment. We apply our schemes to both single-and multi-object dissemination. Our extensive performance study shows that the adaptive mechanisms are effective in a dynamic context and the proposed static tree construction algorithms perform close to optimal in a static environment.

show abstract

Section: Related Workmentioning

confidence: 99%

Disseminating streaming data in a dynamic environment: an adaptive and cost-based approach

2007

View full text Add to dashboard Cite

show abstract

“…In [22], [25] it was shown that multiple trees enable better network utilization and reduce redundant transmissions by having clients become part of only those trees that publish relevant data. XFlow treats query operators as stream subscribers and publishers.…”

Section: Distributed Stream Processingmentioning

confidence: 99%

“…RELATED WORK Data Dissemination. Distributed pub-sub systems have been proposed for dissemination of XML messages [12], [19], [20] or relational data [21], [22]. These systems focus on either reducing the bandwidth usage [12], [22], improving the resource allocation [20], or providing efficient subscription processing mechanisms [19], [21].…”

Section: Operator Replication and Partitionmentioning

confidence: 99%

Supporting Generic Cost Models for Wide-Area Stream Processing

Papaemmanouil

Çetintemel

Jannotti

2009

2009 IEEE 25th International Conference on Data Engineering

View full text Add to dashboard Cite

Abstract-Existing stream processing systems are optimized for a specific metric, which may limit their applicability to diverse applications and environments. This paper presents XFlow, a generic data stream collection, processing, and dissemination system that addresses this limitation efficiently. XFlow can express and optimize a variety of optimization metrics and constraints by distributing stream processing queries across a wide-area network. It uses metric-independent decentralized algorithms that work on localized, aggregated statistics, while avoiding local optima. To facilitate light-weight dynamic changes on the query deployment, XFlow relies on a loosely-coupled, flexible architecture consisting of multiple publish-subscribe overlay trees that can gracefully scale and adapt to changes to network and workload conditions. Based on the desired performance goals, the system progressively refines the query deployment, the structure of the overlay trees, as well as the statistics collection process.We provide an overview of XFlow's architecture and discuss its decentralized optimization model. We demonstrate its flexibility and the effectiveness using real-world streams and experimental results obtained from XFlow's deployment on PlanetLab. The experiments reveal that XFlow can effectively optimize various performance metrics in the presence of varying network and workload conditions.

show abstract

“…The channelization problem formulated in [11] provides a solution to map sources and destinations to a limited set of multicast trees to minimize the unwanted message delivery. Another category of work [12], [13], [14], [15] creates a limited number of multicast trees by proper clustering of user subscription profiles. In the above solutions, filtering is done at the source, at the receiving point, or both.…”

Section: Related Workmentioning

confidence: 99%

A Fast Content-Based Data Distribution Infrastructure

Ganguly

Bhatnagar

Saxena

et al. 2006

Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications

View full text Add to dashboard Cite

Abstract-We present Sieve -an infrastructure for fast content-based data distribution to interested users. The ability of Sieve to filter and forward high-bandwidth data streams stems from its distributed pipelined architecture. The complex message filtering task is broken-up into a sequence of light-weight filtering components resulting in high end-to-end throughput. Furthermore, since each component is assigned to a node based on its resource constraints, the queue buildup inside the nodes is minimal resulting in low end-to-end latency. Our experimental results based on real system implementation show that Sieve can sustain a throughput of more than 5000 messages per second for 100000 subscriptions with predicates of 10 attributes.

show abstract

SemCast: Semantic Multicast for Content-Based Data Dissemination

Abstract: We address the problem of content-based dissemination of highly-distributed, high-volume

Cited by 47 publications

References 24 publications

Disseminating streaming data in a dynamic environment: an adaptive and cost-based approach

Disseminating streaming data in a dynamic environment: an adaptive and cost-based approach

Supporting Generic Cost Models for Wide-Area Stream Processing

A Fast Content-Based Data Distribution Infrastructure

Contact Info

Product

Resources

About