Analysis of Power-Saving Techniques over a Large Multi-use Cluster

McGough, Andrew Stephen; Gerrard, Clive; Noble, Jonathan; Robinson, Paul; Wheater, SM

doi:10.1109/dasc.2011.78

Cited by 12 publications

(21 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Under this situation we seek to record the number of good jobs which have been incorrectly terminated and minimise the number of such circumstances. This reduction can be achieved by looking at the conditions under which jobs have been evicted [59] or providing dedicated resources for jobs which have been evicted [64].…”

Section: Metricsmentioning

confidence: 99%

HTC‐Sim: a trace‐driven simulation framework for energy consumption in high‐throughput computing systems

Forshaw¹,

McGough²,

Thomas³

2016

Concurrency and Computation

View full text Add to dashboard Cite

SUMMARYHigh Throughput Computing (HTC) is a powerful paradigm allowing vast quantities of independent work to be performed simultaneously across many loosely coupled computers. These systems often exploit the idle time available on computers provisioned for other purposes -volunteer computing. However, until recently, little evaluation has been performed on the energy impact of HTC. Many organisations now seek to minimise energy consumption across their IT infrastructure. However, it is unclear how this will affect the usability of HTC systems especially when exploiting volunteer computers. We present here HTC-Sim, a simulation system which allows the evaluation of different energy reduction policies across an HTC system. We model systems which are comprised of both collections of computational resources dedicated to HTC work and resources provided through volunteer computing -a Desktop Grid. We demonstrate that our simulation software scales linearly with increasing HTC workload. We go further to evaluate a number of resource selection policies in terms of the overheads / slowdown incurred, and the energy impact on the HTC system.

show abstract

Section: Metricsmentioning

confidence: 99%

HTC‐Sim: a trace‐driven simulation framework for energy consumption in high‐throughput computing systems

Forshaw¹,

McGough²,

Thomas³

2016

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…C2: Targeting less used computers [6]. By selecting resources with longer idle times between users reduces the chance that a task will be deallocated due to preemption.…”

Section: X0mentioning

confidence: 99%

“…We have extended our cluster based simulation for Condor [6] to take account of the data transfer times. The iperf bandwidth testing software [17] was used to compute the maximum bandwidths available between computers for different payload sizes.…”

Section: Simulation Modelmentioning

confidence: 99%

See 1 more Smart Citation

Reducing the Number of Miscreant Tasks Executions in a Multi-use Cluster

McGough

Forshaw

Gerrard

et al. 2012

2012 Second International Conference on Cloud and Green Computing

Self Cite

View full text Add to dashboard Cite

any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Additional information:Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. Abstract-Exploiting computational resources within an organisation for more than their primary task offers great benefits -making better use of capital expenditure and provides a pool of computational power. This can be achieved through the deployment of a cycle stealing distributed system, where tasks execute during the idle time on computers. However, if a task has not completed when a computer returns to its primary function the task will be preempted, wasting time (and energy), and is often reallocated to a new resource in an attempt to complete. This becomes exacerbated when tasks are incapable of completing due to excessive execution time or faulty hardware / software, leading to a situation where tasks are perpetually reallocated between computers -wasting time and energy. In this work we investigate techniques to increase the chance of 'good' tasks completing whilst curtailing the execution of 'bad' tasks. We demonstrate, through simulation, that we could have reduce the energy consumption of our cycle stealing system by approximately 50%.

show abstract

“…In either case such probability models normally lack the fidelity exhibited within real trace-logs. In prior work [19] we have generated synthetic trace-logs derived from statistical analysis of the HTCondor users at Newcastle University in order to upscale the system load. Although this generated data which matched the statistical characteristics of the original trace-log the newly generated data failed to exhibit the finer-grained characteristics of the original trace data.…”

Section: Introductionmentioning

confidence: 99%

Using Machine Learning in Trace-driven Energy-Aware Simulations of High-Throughput Computing Systems

McGough

Moubayed

Forshaw

2017

Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion

Self Cite

View full text Add to dashboard Cite

Use policyThe full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. ABSTRACTWhen performing a trace-driven simulation of a High Throughput Computing system we are limited to the knowledge which should be available to the system at the current point within the simulation. However, the trace-log contains information we would not be privy to during the simulation. Through the use of Machine Learning we can extract the latent patterns within the trace-log allowing us to accurately predict characteristics of tasks based only on the information we would know. These characteristics will allow us to make better decisions within simulations allowing us to derive better policies for saving energy. We demonstrate that we can accurately predict (up-to 99% accuracy), using oversampling and deep learning, those tasks which will complete while at the same time provide accurate predictions for the task execution time and memory footprint using Random Forest Regression.

show abstract

Analysis of Power-Saving Techniques over a Large Multi-use Cluster

Cited by 12 publications

References 8 publications

HTC‐Sim: a trace‐driven simulation framework for energy consumption in high‐throughput computing systems

HTC‐Sim: a trace‐driven simulation framework for energy consumption in high‐throughput computing systems

Reducing the Number of Miscreant Tasks Executions in a Multi-use Cluster

Using Machine Learning in Trace-driven Energy-Aware Simulations of High-Throughput Computing Systems

Contact Info

Product

Resources

About