On energy management, load balancing and replication

Lang, Willis; Patel, Jignesh M.; Naughton, Jeffrey F.

doi:10.1145/1815948.1815956

Cited by 55 publications

(44 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shutting down a replicated parallel database environ-ment was analyzed in [20]. Other related methods [29,31] either rely on learning request skew, specialized hardware, and data migration.…”

Section: Related Workmentioning

confidence: 99%

Energy management for MapReduce clusters

2010

Self Cite

View full text Add to dashboard Cite

The area of cluster-level energy management has attracted significant research attention over the past few years. One class of techniques to reduce the energy consumption of clusters is to selectively power down nodes during periods of low utilization to increase energy efficiency. One can think of a number of ways of selectively powering down nodes, each with varying impact on the workload response time and overall energy consumption. Since the MapReduce framework is becoming "ubiquitous", the focus of this paper is on developing a framework for systematically considering various MapReduce node power down strategies, and their impact on the overall energy consumption and workload response time.We closely examine two extreme techniques that can be accommodated in this framework. The first is based on a recently proposed technique called "Covering Set" (CS) that keeps only a small fraction of the nodes powered up during periods of low utilization. At the other extreme is a technique that we propose in this paper, called the All-In Strategy (AIS). AIS uses all the nodes in the cluster to run a workload and then powers down the entire cluster. Using both actual evaluation and analytical modeling we bring out the differences between these two extreme techniques and show that AIS is often the right energy saving strategy.

show abstract

Section: Related Workmentioning

confidence: 99%

Energy management for MapReduce clusters

2010

Self Cite

View full text Add to dashboard Cite

show abstract

“…Generally, related work can be classified into two categories based on the approach of achieving energy efficiency. The first category is to seek energy proportionality with non-energyproportional servers [53,32,31,27,35]. The second category is to build more energy-efficient architecture based on low-power CPU [38,41,49,33,43,46,29].…”

Section: Related Workmentioning

confidence: 99%

An experimental evaluation of datacenter workloads on low-power embedded micro servers

Zhao

Shen

et al. 2016

Proc. VLDB Endow.

View full text Add to dashboard Cite

This paper presents a comprehensive evaluation of an ultralow power cluster, built upon the Intel Edison based micro servers. The improved performance and high energy efficiency of micro servers have driven both academia and industry to explore the possibility of replacing conventional brawny servers with a larger swarm of embedded micro servers. Existing attempts mostly focus on mobile-class micro servers, whose capacities are similar to mobile phones. We, on the other hand, target on sensor-class micro servers, which are originally intended for uses in wearable technologies, sensor networks, and Internet-of-Things. Although sensor-class micro servers have much less capacity, they are touted for minimal power consumption (< 1 Watt), which opens new possibilities of achieving higher energy efficiency in datacenter workloads. Our systematic evaluation of the Edison cluster and comparisons to conventional brawny clusters involve careful workload choosing and laborious parameter tuning, which ensures maximum server utilization and thus fair comparisons. Results show that the Edison cluster achieves up to 3.5× improvement on work-done-per-joule for web service applications and data-intensive MapReduce jobs. In terms of scalability, the Edison cluster scales linearly on the throughput of web service workloads, and also shows satisfactory scalability for MapReduce workloads despite coordination overhead.

show abstract

“…For simplicity, we assume two-degree replication in the following without losing generality. In the case of two-degree replication as shown in Table 2, operator o 1 in the 1 st degree of replica's ordered list has the largest weight and operator o 2 in the 2 nd degree of replica's ordered list has the second largest weight, then we combine operator o 1 and operator o 2 as a super-operator O 12 .…”

Section: Super Operator Based On Multi-degree Operator Replicasmentioning

confidence: 99%

“…However, such an approach does not regard energy consumption. Although there are many resources over the wide area network, we should take energy consumption into consideration that is a crucial and rising problem in the real world [12], namely, fully utilizing the network resources for decreasing resources waste. The other is the load shedding method [13]- [15] that drops part of data once servers are overloaded during the data stream processing.…”

Section: Introductionmentioning

confidence: 99%

Economical and Fault-Tolerant Load Balancing in Distributed Stream Processing Systems

Xiao

Kitasuka

Aritsugi

2012

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYWe present an economical and fault-tolerant load balancing strategy (EFTLBS) based on an operator replication mechanism and a load shedding method, that fully utilizes the network resources to realize continuous and highly-available data stream processing without dynamic operator migration over wide area networks. In this paper, we first design an economical operator distribution (EOD) plan based on a binpacking model under the constraints of each stream bandwidth as well as each server's CPU capacity. Next, we devise super-operator (SO) that load balances multi-degree operator replicas. Moreover, for improving the faulttolerance of the system, we color the SOs based on a coloring bin-packing (CBP) model that assigns peer operator replicas to different servers. To minimize the effects of input rate bursts upon the system, we take advantage of a load shedding method while keeping the QoS guarantees made by the system based on the SO scheme and the CBP model. Finally, we substantiate the utility of our work through experiments on ns-3. key words:distributed data stream processing systems (DDSPSs), load balancing strategy, fault-tolerance, economical operator distribution (EOD)

show abstract

On energy management, load balancing and replication

Cited by 55 publications

References 20 publications

Energy management for MapReduce clusters

Energy management for MapReduce clusters

An experimental evaluation of datacenter workloads on low-power embedded micro servers

Economical and Fault-Tolerant Load Balancing in Distributed Stream Processing Systems

Contact Info

Product

Resources

About