Adaptive component composition and load balancing for distributed stream processing applications

Abstract

Providing real-time and QoS support to stream processing applications running on top of largescale overlays is challenging due to the inherent heterogeneity and resource limitations of the nodes and the multiple QoS demands of the applications that must concurrently be met. In this paper we propose an integrated adaptive component composition and load balancing mechanism that (1) allows the composition of distributed stream processing applications on the fly across a large-scale system, while satisfying their QoS demands and distributing the load fairly on the resources, and (2) adapts dynamically to changes in the resource utilization or the QoS requirements of the applications. Our extensive experimental results using both simulations as well as a prototype deployment illustrate the efficiency, performance and scalability of our approach.

Sensor Service Selection through Switch Options

“…In more detail, [3] and [4] consider only energy spending, while [5] deals only with load balancing issues. Although [6] does take into account different metrics, possible combinations of them are ignored.…”

Resource Allocation, Trading and Adaptation in Self-managing Systems

“…The scientific literature proposes a wide range of solutions for resource management in various application domains, which range from aeronautics [21] to telecommunications [5], from grid computing [10] to SOA [4,17] and Web Services [3], from multimedia [18] to multi-agent systems. Typically, they focus on one or two functionalities offered by the framework proposed in this paper.…”

Energy-Reduction Offloading Technique for Streaming Media Servers

“…Already today multimedia streaming takes up about 80% of video data traffic and engages for even bigger share in future. Heterogeneity of modern mobile devices, that is the difference in screen size and computational power of each concrete model of mobile devices along with their energy limitations, necessitates customized video transcoding for all end user devices with the unique parameters [1,2]. Also, different devices can have different Internet bandwidth, which imposes additional requirements upon the quality of streaming video and therefore customized transcoding [3].…”

“…So far, the existing researches concern moving of the transcoding process, the most computation-intensive part of video streaming, to other accessible resources. In [2] transcoding is done by adaptation engine which is a solution for multimedia data customization problem rather than the media server overload problem. In [6,7] adaptive transcoding performed at proxy servers or video adaptation nodes located on the edges of fixed networks, while in [8,9] transcoding is fairly spread among existing neighbor servers.…”