Distributed Joins and Data Placement for Minimal Network Traffic

Polychroniou, Orestis; Zhang, Wangda; Ross, Kenneth A.

doi:10.1145/3241039

Cited by 21 publications

(46 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Query planning. Database research has explored query planning and optimization extensively [4,32,36]. Gigascope performs query partitioning to minimize the data transfer from the capture card to the stream processor [10].…”

Section: Related Workmentioning

confidence: 99%

“…Gigascope performs query partitioning to minimize the data transfer from the capture card to the stream processor [10]. Sensor networks have explored the query partitioning problems that are similar to those that Sonata faces [4,27,28,32,36,46]. However, these systems face different optimization problems because they typically involve lower traffic rates and involve special-purpose queries.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Sonata

Gupta

Harrison

Canini

et al. 2018

Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication

195

View full text Add to dashboard Cite

Managing and securing networks requires collecting and analyzing network traffic data in real time. Existing telemetry systems do not allow operators to express the range of queries needed to perform management or scale to large traffic volumes and rates. We present Sonata, an expressive and scalable telemetry system that coordinates joint collection and analysis of network traffic. Sonata provides a declarative interface to express queries for a wide range of common telemetry tasks; to enable real-time execution, Sonata partitions each query across the stream processor and the data plane, running as much of the query as it can on the network switch, at line rate. To optimize the use of limited switch memory, Sonata dynamically refines each query to ensure that available resources focus only on traffic that satisfies the query. Our evaluation shows that Sonata can support a wide range of telemetry tasks while reducing the workload for the stream processor by as much as seven orders of magnitude compared to existing telemetry systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Sonata

Gupta

Harrison

Canini

et al. 2018

Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication

195

View full text Add to dashboard Cite

show abstract

“…Network operators have leveraged the recent advances in the area of scalable streaming data analysis [2,16,24,29,32] to build platforms capable of processing network data at very high rates. The database community has also explored the query optimization problem extensively [3,27]. Gigascope [13] uses query partitioning to minimize the data transfer within the stream processor.…”

Section: Related Workmentioning

confidence: 99%

Network Monitoring as a Streaming Analytics Problem

Gupta

Birkner

Canini

et al. 2016

Proceedings of the 15th ACM Workshop on Hot Topics in Networks

View full text Add to dashboard Cite

Programmable switches make it easier to perform flexible network monitoring queries at line rate, and scalable stream processors make it possible to fuse data streams to answer more sophisticated queries about the network in real-time. Unfortunately, processing such network monitoring queries at high traffic rates requires both the switches and the stream processors to filter the traffic iteratively and adaptively so as to extract only that traffic that is of interest to the query at hand. Others have network monitoring in the context of streaming; yet, previous work has not closed the loop in a way that allows network operators to perform streaming analytics for network monitoring applications at scale. To achieve this objective, Sonata allows operators to express a network monitoring query by considering each packet as a tuple and efficiently partitioning each query between the switches and the stream processor through iterative refinement. Sonata extracts only the traffic that pertains to each query, ensuring that the stream processor can scale traffic rates of several terabits per second. We show with a simple example query involving DNS reflection attacks and traffic traces from one of the world's largest IXPs that Sonata can capture 95% of all traffic pertaining to the query, while reducing the overall data rate by a factor of about 400 and the number of required counters by four orders of magnitude.

show abstract

“…Therefore, retrieving the global skylines from all the distributed local sites with minimum communication overhead in these scenarios will be an important and challenging problem, because of the consideration for the introduction of the bandwidth consumption and network delay. Until recently, there appear some works that target at distributed queries over uncertain data, such as top‐ k dominating and join queries …”

Section: Introductionmentioning

confidence: 99%

“…Given three 2-dimensional interval objects a = {[10,20],[20,40]}, b = {[12,16],[10,30]}, and c = {[8,22],[28,32]} (see…”

mentioning

confidence: 99%

Efficient skyline computation over distributed interval data

Ren

2017

Concurrency and Computation

View full text Add to dashboard Cite

Summary The increasing volume of uncertain data has resulted in a dire need for supporting efficient uncertain data management. The skyline query as an important aspect of data management has received considerable attention in recent years, because of its importance in making intelligent decisions over complex data. Moreover, data collection and storage have become increasingly distributed, which makes the central assembly of data for storage and query infeasible and inefficient. Although many research efforts have been conducted to address the skyline query problem in various distributed scenarios, we still lack algorithms to address the queries over interval data, which is a special kind of attribute‐level uncertain data that widely exists in many applications. In this paper, we extensively study the skyline query over distributed interval data. We model the skyline query problem and define the distributed skyline query over interval data. Particularly, 2 efficient algorithms are proposed to retrieve the skylines progressively from distributed local sites with a highly optimized feedback framework. Moreover, we exploit 2 strategies for further improving the queries. Extensive experiments on synthetic and real datasets with real deployment are conducted to validate the effectiveness and efficiency of our proposals.

show abstract

Distributed Joins and Data Placement for Minimal Network Traffic

Cited by 21 publications

References 62 publications

Sonata

Sonata

Network Monitoring as a Streaming Analytics Problem

Efficient skyline computation over distributed interval data

Contact Info

Product

Resources

About