Quantifying I/O and Communication Traffic Interference on Dragonfly Networks Equipped with Burst Buffers

Mubarak, Misbah; Carns, Philip; Jenkins, John; Li, Jianping Kelvin; Jain, Nikhil; Snyder, Shane A.; Ross, Robert; Carothers, Christopher D.; Bhatele, Abhinav; Ma, Kwan‐Liu

doi:10.1109/cluster.2017.25

Cited by 25 publications

(12 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A typical architecture consists in locating the Burst-Buffers between the compute nodes and the Parallel File System (PFS). This is the case of DDN IME [8], [14] and Cray DataWarp [9], [15], [16]. In this pseudo-centralized architecture, the Burst-Buffers are often colocated with the I/O nodes.…”

Section: A Burst-buffers Architectures and Modelsmentioning

confidence: 99%

“…Several management strategies have been proposed. Mubarak et al [15] study the case where the buffers are shared between the different applications running onto the platform and used both to accelerate transfers and to prevent I/O congestion. On the contrary, in Schenck et al [14] and Daley et al [16], applications decide the size of the buffer that should be dedicated to them.…”

Section: A Burst-buffers Architectures and Modelsmentioning

confidence: 99%

“…Other strategies focus on how to share them between the different applications [17], [18]. In [15], the interaction between the placement of Burst-Buffers and high-radix interconnect topologies is studied. In the context of fault-tolerance, using a buffer on a different node can allow the implementation of hierarchical checkpointing strategies that provide more resilience than in-node buffer strategies [18].…”

Section: A Burst-buffers Architectures and Modelsmentioning

confidence: 99%

See 2 more Smart Citations

Sizing and Partitioning Strategies for Burst-Buffers to Reduce IO Contention

Aupy¹,

Beaumont²,

Eyraud-Dubois³

2019

2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

Burst-Buffers are high throughput and small size storage which are being used as an intermediate storage between the PFS (Parallel File System) and the computational nodes of modern HPC systems. They can allow to hinder to contention to the PFS, a shared resource whose read and write performance increase slower than processing power in HPC systems. A second usage is to accelerate data transfers and to hide the latency to the PFS. In this paper, we concentrate on the first usage. We propose a model for Burst-Buffers and application transfers.We consider the problem of dimensioning and sharing the Burst-Buffers between several applications. This dimensioning can be done either dynamically or statically. The dynamic allocation considers that any application can use any available portion of the Burst-Buffers. The static allocation considers that when a new application enters the system, it is assigned some portion of the Burst-Buffers, which cannot be used by the other applications until that application leaves the system and its data is purged from it. We show that the general sharing problem to guarantee fair performance for all applications is an NP-Complete problem. We propose a polynomial time algorithms for the special case of finding the optimal buffer size such that no application is slowed down due to PFS contention, both in the static and dynamic cases. Finally, we provide evaluations of our algorithms in realistic settings. We use those to discuss how to minimize the overhead of the static allocation of buffers compared to the dynamic allocation.

show abstract

Section: A Burst-buffers Architectures and Modelsmentioning

confidence: 99%

Section: A Burst-buffers Architectures and Modelsmentioning

confidence: 99%

Section: A Burst-buffers Architectures and Modelsmentioning

confidence: 99%

See 1 more Smart Citation

Sizing and Partitioning Strategies for Burst-Buffers to Reduce IO Contention

Aupy¹,

Beaumont²,

Eyraud-Dubois³

2019

2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

show abstract

“…Typically, Burst-Buffers can be located between the compute nodes and the Parallel File System (PFS). This is the case of DDN IME [7], [10] and Cray DataWarp [8], [11], [9]. In this pseudo-centralized architecture, the Burst-Buffers are often colocated with the I/O nodes.…”

Section: Related Work a Burst-buffer Architectures And Modelsmentioning

confidence: 99%

What Size Should Your Buffers to Disks be?

Aupy

Beaumont

Eyraud-Dubois

2018

2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

View full text Add to dashboard Cite

Burst-Buffers are high throughput, small size intermediate storage systems typically based on SSDs or NVRAM that are designed to be used as a potential buffer between the computing nodes of a supercomputer and its main storage system consisting of hard drives. Their purpose is to absorb the bursts of I/O that many HPC applications experience (for example for saving checkpoints or data from intermediate results). In this paper, we propose a probabilistic model for evaluating the performance of Burst-Buffers. From a model of application and a data management strategy, we build a Markov-chain-based model of the system, that allows us to quickly answer issues about dimensioning of the system: for a given set of applications, and for a given Burst-Buffer size and bandwidth, how often does the buffer overflow? We also provide extensive simulation results to validate our modeling approach.

show abstract

“…Despite years of innovation in network routing, congestion avoidance, and mitigation algorithms across generation highperformance interconnects, extreme-scale applications running on high-performance computing systems continue to suffer from performance variation and scaling challenges [1]- [3] due to (i) frequent exposure to congestion; and (ii) the inability to automatically optimize resource parameters (such as placement, rank mapping, and application scheduling) to improve network utilization. The current interconnects suffer from congestion that can occur because of (i) bad application placement (e.g., tightly packed ranks versus ranks spread across the network) [4], [5] and presence of bully applications [6]; (ii) the presence of large numbers of failed links [7] and new link failures, which force adversarial traffic shaping/flow on the network; and (iii) inherent congestion susceptibility due to network design choices (e.g., directional order routing in torus networks) or design bugs/gaps (e.g., incorrect/bad dynamic routing policy).…”

Section: Introductionmentioning

confidence: 99%

A Study of Network Congestion in Two Supercomputing High-Speed Interconnects

Jha

Patke

Brandt

et al. 2019

2019 IEEE Symposium on High-Performance Interconnects (HOTI)

View full text Add to dashboard Cite

Network congestion in high-speed interconnects is a major source of application runtime performance variation. Recent years have witnessed a surge of interest from both academia and industry in the development of novel approaches for congestion control at the network level and in application placement, mapping, and scheduling at the system-level. However, these studies are based on proxy applications and benchmarks that are not representative of field-congestion characteristics of high-speed interconnects. To address this gap, we present (a) an end-to-end framework for monitoring and analysis to support long-term field-congestion characterization studies, and (b) an empirical study of network congestion in petascale systems across two different interconnect technologies: (i) Cray Gemini, which uses a 3-D torus topology, and (ii) Cray Aries, which uses the DragonFly topology.

show abstract

Quantifying I/O and Communication Traffic Interference on Dragonfly Networks Equipped with Burst Buffers

Cited by 25 publications

References 26 publications

Sizing and Partitioning Strategies for Burst-Buffers to Reduce IO Contention

Sizing and Partitioning Strategies for Burst-Buffers to Reduce IO Contention

What Size Should Your Buffers to Disks be?

A Study of Network Congestion in Two Supercomputing High-Speed Interconnects

Contact Info

Product

Resources

About