On SDN-Enabled Online and Dynamic Bandwidth Allocation for Stream Analytics

Aljoby, Walid; Wang, Xin; Fu, Tom Z. J.; Richard, T. B.

doi:10.1109/icnp.2018.00030

Cited by 9 publications

(12 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Urban users send data to the platform via cellular networks and remote users via satellite-terrestrial network. The backbone network transmits the data to the SDN controller center [41], which consists of databases and servers for analyzing and processing the sensing data returned by users to set up smart sensing systems powered by different applications.…”

Section: System Modelmentioning

confidence: 99%

A Privacy-Preserving Incentive Mechanism for Data Offloading in Satellite-Terrestrial Crowdsensing

Zhu

Liu

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Data offloading algorithm is the foundation of urban Internet of Things, which has gained attention for its large size of user engagement, low cost, and wide range of data sources, replacing traditional crowdsensing in areas such as intelligent vehicles, spectrum sensing, and environmental surveillance. In data offloading tasks, users’ location information is usually required for optimal task assignment, while some users in remote areas are unable to access base station signals, making them incapable of performing sensing tasks, and at the same time, there are serious concerns about users’ privacy leakage about their locations. Until today, location protection for task assignment in data offloading has not been well explored. In addition, existing privacy protection algorithms and data offloading task assignment mechanisms cannot provide personalized protection for different users’ privacy protection needs. To this end, we propose an algorithm known as differential private long-term privacy-preserving auction with Lyapunov stochastic theory (DP-LAL) for data offloading based on satellite-terrestrial architecture that minimizes the total payment. This not only gives an approximate optimal total payment in polynomial time but also improves the issue of poor signal in remote areas. Meanwhile, satellite-terrestrial data offloading architecture integrates wireless sensor networks and cloud computing to provide real-time data processing. What is more, we have considered long-term privacy protection goals. We employ reverse combinatorial auction and Lyapunov optimization theorem to jointly optimize queue stability and total payment. More importantly, we use Lyapunov optimization theorem to jointly optimize queue stability and total payment. We prove that our algorithm is of high efficiency in computing and has good performance in various economic attributes. For example, our algorithms are personally rational, budget-balanced, and true to the buyer and seller. We use large-scale simulations to evaluate the proposed algorithm, and compare our algorithm with existing algorithms, our algorithm shows higher efficiency and better economic properties.

show abstract

Section: System Modelmentioning

confidence: 99%

A Privacy-Preserving Incentive Mechanism for Data Offloading in Satellite-Terrestrial Crowdsensing

Zhu

Liu

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Therefore, in this paper, we rigorously develop max-min fair utility functions and use novel metrics of flow urgency. 1 for bandwidth, such that to entitle bandwidth allocation to the optimality as driven by application layer. Recall that such metrics are periodically retrieved such that to allow our optimization to work online over the lifetime of long-running stream analytics.…”

Section: Motivation Examplementioning

confidence: 99%

On SDN-Enabled Online and Dynamic Bandwidth Allocation for Stream Analytics

Aljoby

Wang

et al. 2019

IEEE J. Select. Areas Commun.

Self Cite

View full text Add to dashboard Cite

Data communication in cloud-based distributed stream data analytics often involves a collection of parallel and pipelined TCP flows. As the standard TCP congestion control mechanism and its variants are designed for achieving "fairness" among competing flows and are agnostic to the application layer contexts, the bandwidth allocation among a set of TCP flows traversing bottleneck links often leads to sub-optimal application-layer performance measures, e.g., stream processing throughput or average tuple complete latency. Motivated by this and enabled by the rapid development of the software-defined networking (SDN) techniques, in this paper, we re-investigate the design space of the bandwidth allocation problem and propose a cross-layer framework which utilizes the instantaneous information obtained from the application layer and provides on-thefly and dynamic bandwidth adjustment algorithms for assisting the stream analytics applications achieving better performance during the runtime. We implement a prototype cross-layer bandwidth allocation framework based on a popular open-source distributed stream processing platform, Apache Storm, together with the OpenDaylight controller, and carry out extensive experiments with real-world analytical workloads on top of a local cluster consisting of ten workstations interconnected by a SDN-enabled fat-tree like testbed. The experiment results clearly validate the effectiveness and efficiency of our proposed framework and algorithms. Finally, we leverage the proposed cross-layer SDN framework and introduce an exemplary mechanism for bandwidth sharing and performance reasoning among multiple active applications and show a case of a point solution on how to approximate application-level fairness. optimizing network activity is important to improving and delivering real-time responses in these applications. In this context, there has been flurry of research attempts toward optimizing streaming applications. While in large part successful, however, their focus mainly has centered to schedule and provision computation resources of the applications or limited to minimizing traffic across the network. Hence, these solutions have largely overlooked allocation and provision of network bandwidth. As a result, they are either suboptimal in optimizing network transfer [9], [11], [12], or assuming the network with sufficient bandwidth resource [13].In current stream processing frameworks, the share of network bandwidth has left to the mercy of the underlying transport mechanisms (e.g., TCP [14], DCTCP [15]).Nonetheless, such mechanisms are designed mainly for end-to-end data delivery in an application agnostic manner,

show abstract

“…This paper aims at developing a framework for efficient transport protocol design and optimization of the DEL. Data parallelism is the most widely used technique in DEL for its convenience [7], [8]. As shown in Fig.…”

Section: Introductionmentioning

confidence: 99%

“…1, the difference in data parallelism schemes relies on the different tolerances for their parameter inconsistencies. As reported in [7], [8], other prior mitigation solutions have significant limitations. They balance data flow rate from workers across paths but often incur backlogs (and bufferbloat) when the paths exhibit variance, causing stragglers.…”

Section: Introductionmentioning

confidence: 99%