Introduction to Energy Efficiency in Large-Scale Distributed Systems

Pierson, Jean‐Marc; Hlavacs, Helmut

doi:10.1002/9781118981122.ch1

Cited by 5 publications

(5 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For an NVIDIA Tesla P100 with 4 NVLINKs, 16 lanes/link, we thus assume each lane consumes ≈ 0.7 watts and each link consumes ≈ 0.7×16 = 11.2 watts 39 . We also assume that NVLINK and PCIe consume 85% of its peak energy when they have no load (similar to the reports in other works 37,38 36 an Omni‐path spine, 40 and an edge switch 41 to their number of ports, respectively.…”

Section: Discussionmentioning

confidence: 90%

“…Instead, we only consider those of network devices including switches and links. We use the simplest power model mentioned in the work of Dayarathna et al 37 that is integrated into Simgrid as in the work of Amersho 38 . In this model, the power consumption of a given device for a time duration T can be expressed as

({, \begin{matrix} array \\ array E (T, φ) = \int_{0}^{T} (P_{idle} + P_{dynamic, φ}) (t) d t \\ array P_{dynamic, φ} = (P_{peak} - P_{idle}) \times φ \end{matrix}),

where φ is the normalized link utilization obtained by dividing the link load to its bandwidth.…”

Section: Discussionmentioning

confidence: 99%

“…39 We also assume that NVLINK and PCIe consume 85% of its peak energy when they have no load (similar to the reports in other works. 37,38 ) We also get the peak power and idle power per port of a PCIe, a spine, and a leaf switch by dividing their maximum and typical power consumption of a PCI express gen 3 switch, 36 an Omni-path spine, 40 and an edge switch 41 to their number of ports, respectively. Table 4 shows our detailed configuration in this evaluation.…”

Section: Network Power Consumptionmentioning

confidence: 99%

“…We use the simplest power model mentioned in the work of Dayarathna et al 37 that is integrated into Simgrid as in the work of Amersho. 38 In this model, the power consumption of a given device for a time duration T can be expressed as…”

Section: Network Power Consumptionmentioning

confidence: 99%

See 3 more Smart Citations

Efficient MPI‐AllReduce for large‐scale deep learning on GPU‐clusters

Nguyen

Wahib

Takano

2019

Concurrency and Computation

View full text Add to dashboard Cite

Training models on large-scale GPUs-accelerated clusters are becoming a commonplace due to the increase in complexity and size in Deep Learning models. One of the main challenges for distributed training is the collective communication overhead for large message sizes: up to hundreds of MB. In this paper, we propose two hierarchical distributed memory multi-leader allreduce algorithms optimized for GPU-accelerated clusters (named lr_lr and lr_rab). In which, GPUs inside a computing node perform an intra-node communication phase to gather and store results of local reduced values to designated GPUs (known as node leaders). Node leaders then keep a role as an inter-node communicator. Each leader exchanges one part of reduced values to the leaders of the other nodes in parallel. Hence, we are capable of significantly reducing the time for injecting data into the inter-node network. We also overlap the inter-node and intra-node communication by implementing our proposal in a pipelined manner. We evaluate those algorithms on the discrete-event simulation Simgrid. We show that our algorithms, lr_lr and lr_rab, can cut down the execution time of an Allreduce micro-benchmark that uses the logical ring algorithm (lr) by up to 45% and 51%, respectively. With the pipelined implementation, our lr_lr_pipe achieve 15% performance improvement when compared to lr_lr. In addition, simulation result also project power savings for the network devices of up to 23% and 32%.

show abstract

Section: Discussionmentioning

confidence: 90%

({, \begin{matrix} array \\ array E (T, φ) = \int_{0}^{T} (P_{idle} + P_{dynamic, φ}) (t) d t \\ array P_{dynamic, φ} = (P_{peak} - P_{idle}) \times φ \end{matrix}),

where φ is the normalized link utilization obtained by dividing the link load to its bandwidth.…”

Section: Discussionmentioning

confidence: 99%

Section: Network Power Consumptionmentioning

confidence: 99%

Section: Network Power Consumptionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient MPI‐AllReduce for large‐scale deep learning on GPU‐clusters

Nguyen

Wahib

Takano

2019

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…The latest in-depth survey commissioned by the European Union in 2010 about energy consumption and efficiency of equipment in buildings shows a continuous growth of energy consumption of computer end-use equipments (amongst others) over the last years. Additionally, the world energy consumption for servers has doubled over the period from 2000 to 2005 (IC0804, 2007) (Pierson and Hlavacs, 2005).…”

Section: Department Of Energy 2007)mentioning

confidence: 99%

ICT and power: new challenges and solutions

Dimitrijević

Milojkovic

Bojanić

et al. 2013

IJRIS

View full text Add to dashboard Cite

An attempt is made in this paper to summarise the state of the art in the interaction between the Information Communication Technologies (ICT) that becomes ubiquitous and the electrical power production and distribution. Both are considered to have a difficult task to fulfil enormous rise of demand that is becoming above all expectations. In the same time they technologically interfere in the sense that they mutually help in the fulfilment of their main task while in the same time loading each other with problems inherent to the respective technology. In the paper, we will try to merge our (LEDA laboratory of the University of Niš) own results with ones available in the literature in order to give as complete a picture of the subject as possible. from Faculty of Electronic Engineering Niš. His research interests are virtual instrumentation, computer networks and grid computing. He is a member of IEEE.Jelena Milojković is Researcher at the Faculty of Electronic Engineering of the University of Niš, Serbia. Her research interest is related to sustainable design in electronica, application of artificial neural network and prediction.Slobodan Bojanić is Professor at the ETSI Telecomunicación, Universidad Politécnica de Madrid, Spain. His main research interest is related to information security, cryptography, electronics for security applications.Octavio Nieto-Taladriz is Professor at the ETSI Telecomunicación, Universidad Politécnica de Madrid, Spain. His main research interest is related to microelectronics, embedded systems, ambient intelligence, sensor networks, ubiquitous computing, security. He entered in the Order of Policy Merit and received the Cross of Policy Merit with White Distinction awarded by Ministry of the Interior for his research career in the area of electronic and telecommunication systems for the security of citizens.Vančo Litovski is a Professor at the University of Niš since 1987. His research interest spans over several fields such as: electronic filter design, VLSI design, electronic modelling and ICT and power 33 simulation, design of measurement equipment for power applications, sustainable ICT, etc. He is member of the Academy of Engineering Sciences of Serbia and President of Yugoslav Simulation Society. He was awarded the Savastano Award, The Tesla Award and the ETRAN Award.This paper is a revised and expanded version of a paper entitled 'ICT and Power -Synergy and Hostility' presented at

show abstract

Energy: A New Criteria for Performances in Large Scale Distributed Systems

Pierson

2011

Performance Evaluation of Computer and Communication Systems. Milestones and Future Challenges

Self Cite

View full text Add to dashboard Cite

Abstract. In the framework of the workshop "Performance Evaluation of Computer and Communication Systems: Milestones and Future Challenges", this paper proposes to address the emerging criteria of energy. Used since a long time in embedded systems where battery operated devices needed a careful handling, the energy metric is taking a large momentum in the last years on large scale systems where thousands of nodes collaborate to serve high end infrastructures like web servers, clouds and grids.

show abstract

Introduction to Energy Efficiency in Large-Scale Distributed Systems

Cited by 5 publications

References 20 publications

Efficient MPI‐AllReduce for large‐scale deep learning on GPU‐clusters

Efficient MPI‐AllReduce for large‐scale deep learning on GPU‐clusters

ICT and power: new challenges and solutions

Energy: A New Criteria for Performances in Large Scale Distributed Systems

Contact Info

Product

Resources

About