Exploring power behaviors and trade-offs of in-situ data analytics

Gamell, Marc; Rodero, Iván; Parashar, Manish; Bennett, Janine Camille; Kolla, Hemanth; Chen, Jacqueline; Bremer, Peer-Timo; Landge, Aaditya G.; Gyulassy, Attila; McCormick, Patrick; Pakin, Scott; Pascucci, Valerio; Klasky, Scott

doi:10.1145/2503210.2503303

Cited by 29 publications

(14 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…While most studies have been focusing on profiling and characterizing power usage in supercomputers, modeling and exploring data-related energy/performance tradeoffs [7], and exploiting dynamic voltage frequency scaling (DVFS) techniques to reduce power consumption [9], there is comparatively little work on investigating the impact of I/O management on energy consumption (i.e., how much energy a supercomputer consumes while running a scientific simulation when adopting different data management approaches).…”

Section: Introductionmentioning

confidence: 99%

A performance and energy analysis of I/O management approaches for exascale systems

Yildiz

Dorier

Ibrahim

et al. 2014

Proceedings of the Sixth International Workshop on Data Intensive Distributed Computing

View full text Add to dashboard Cite

The advent of fast, unprecedentedly scalable, yet energyhungry exascale supercomputers poses a major challenge consisting in sustaining a high performance per watt ratio. While much recent work has explored new approaches to I/O management, aiming to reduce the I/O performance bottleneck exhibited by HPC applications (and hence to improve application performance), there is comparatively little work investigating the impact of I/O management approaches on energy consumption.In this work, we explore how much energy a supercomputer consumes while running scientific simulations when adopting various I/O management approaches. We closely examine three radically different I/O schemes including time partitioning, dedicated cores, and dedicated nodes. We implement the three approaches within the Damaris I/O middleware and perform extensive experiments with one of the target HPC applications of the Blue Waters sustained-petaflop supercomputer project: the CM1 atmospheric model. Our experimental results obtained on the French Grid'5000 platform highlight the differences between these three approaches and illustrate in which way various configurations of the application and of the system can impact performance and energy consumption.

show abstract

Section: Introductionmentioning

confidence: 99%

A performance and energy analysis of I/O management approaches for exascale systems

Yildiz

Dorier

Ibrahim

et al. 2014

Proceedings of the Sixth International Workshop on Data Intensive Distributed Computing

View full text Add to dashboard Cite

show abstract

“…Kamil et al [26] investigate various power measurement methodologies such as line meters, clamp meters, integrated meters and power panels, and opt for power panels in their work. Other methods can be applied, including voltage regulator models that provide current and voltage readings at node level [35], cluster specifications [24], simulators [38] and wattmeters [37]. While there is a wide range of options for measuring the power, most of them are subject to measurement errors.…”

Section: Power Measurement Methodsmentioning

confidence: 99%

“…Gamell et al [24] provide a power model for the in situ analysis of the S3D turbulent combustion code. They investigate the roles of the system's architecture, the algorithm design and various deployment options.…”

Section: Profiling Energy Consumption Of Hpc Simulationsmentioning

confidence: 99%

“…Most of the studies on energy efficiency in HPC target the entire system's power. Gamell et al [24] separate the network component in their model, however they make assumptions for the related component type since power information for every component (in particular the NIC) is not available. Son et al [38] target the power usage of disks.…”

Section: Saving Energymentioning

confidence: 99%

See 1 more Smart Citation

On the energy footprint of I/O management in Exascale HPC systems

Dorier

Yildiz

Ibrahim

et al. 2016

Future Generation Computer Systems

View full text Add to dashboard Cite

International audienceThe advent of unprecedentedly scalable yet energy hungry Exascale supercomputers poses a major challenge in sustaining a high performance-per-watt ratio. With I/O management acquiring a crucial role in supporting scientific simulations, various I/O management approaches have been proposed to achieve high performance and scalability. However, the details of how these approaches affect energy consumption have not been studied yet. Therefore, this paper aims to explore how much energy a supercomputer consumes while running scientific simulations when adopting various I/O management approaches. In particular, we closely examine three radically different I/O schemes including time partitioning, dedicated cores, and dedicated nodes. To do so, we implement the three approaches within the Damaris I/O middleware and perform extensive experiments with one of the target HPC applications of the Blue Waters sustained-petaflop supercomputer project: the CM1 atmospheric model. Our experimental results obtained on the French Grid’5000 platform highlight the differences among these three approaches and illustrate in which way various configurations of the application and of the system can impact performance and energy consumption. Moreover, we propose and validate a mathematical model that estimates the energy consumption of a HPC simulation under different I/O approaches. Our proposed model gives hints to pre-select the most energy-efficient I/O approach for a particular simulation on a particular HPC system and therefore provides a step towards energy-efficient HPC simulations in Exascale systems. To the best of our knowledge, our work provides the first in-depth look into the energy-performance tradeoffs of I/O management approaches

show abstract

“…These challenges are quickly limiting the ability to effectively transform data into insights, and require rethinking traditional data analytics pipelines to reduce data movement. Recently, in-situ and in-transit data processing pipelines have emerged as a promising approach [3] to effectively reduce overheads [4] and energy costs [5] due to data movement by placing data processing operations closer to where the data is being produced.…”

Section: Introductionmentioning

confidence: 99%

In-situ feature-based objects tracking for data-intensive scientific and enterprise analytics workflows

et al. 2014

Self Cite

View full text Add to dashboard Cite

Emerging scientific simulations on leadership class systems are generating huge amounts of data and processing this data in an efficient and timely manner is critical for generating insights from the simulations. However, the increasing gap between computation and disk I/O speeds makes traditional data analytics pipelines based on post-processing cost prohibitive and often infeasible. In this paper, we investigate an alternate approach that aims to bring the analytics closer to the data using in-situ execution of data analysis operations. Specifically, we present the design, implementation and evaluation of a framework that can support in-situ feature-based objects tracking on distributed scientific datasets. Central to this framework is a scalable decentralized and online clustering, a cluster tracking algorithm, which executes in-situ (on different cores) in parallel with the simulation processes, and retrieves data from the simulations directly via on-chip shared memory. The results from our experimental evaluation demonstrate that the in-situ approach significantly reduces the cost of data movement, that the presented framework can support scalable featurebased objects tracking, and that it can be effectively used for in-situ analytics in large scale simulations.

show abstract

Exploring power behaviors and trade-offs of in-situ data analytics

Cited by 29 publications

References 76 publications

A performance and energy analysis of I/O management approaches for exascale systems

A performance and energy analysis of I/O management approaches for exascale systems

On the energy footprint of I/O management in Exascale HPC systems

In-situ feature-based objects tracking for data-intensive scientific and enterprise analytics workflows

Contact Info

Product

Resources

About