Source level transformations to improve I/O data partitioning

Wang, Yijian; Kaeli, David

doi:10.1145/1162618.1162622

Cited by 7 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a similar vein to [28,29], in which I/O throughput is vastly improved by transparently partitioning a data file (creating multiple, independent I/O streams), PLFS uses file partitioning as well as a log-structured file system to further improve the potential I/O bandwidth. Through our tracing tools, we offer an in-depth analysis of the benefits offered by PLFS (Section 6.3).…”

Section: Virtual File Systemsmentioning

confidence: 99%

Parallel File System Analysis Through Application I/O Tracing

Wright

Hammond

Pennycook

et al. 2012

The Computer Journal

View full text Add to dashboard Cite

Input/Output (I/O) operations can represent a significant proportion of the run-time of parallel scientific computing applications. Although there have been several advances in file format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and the compute clusters that they support. In this paper, we document the design and application of the RIOT I/O toolkit (RIOT) being developed at the University of Warwick with our industrial partners at the Atomic Weapons Establishment and Sandia National Laboratories. We use the toolkit to assess the performance of three industry-standard I/O benchmarks on three contrasting supercomputers, ranging from a mid-sized commodity cluster to a large-scale proprietary IBM BlueGene/P system. RIOT provides a powerful framework in which to analyse I/O and parallel file system behaviour-we demonstrate, for example, the large file locking overhead of IBM's General Parallel File System, which can consume nearly 30% of the total write time in the FLASH-IO benchmark. Through I/O trace analysis, we also assess the performance of HDF-5 in its default configuration, identifying a bottleneck created by the use of suboptimal Message Passing Interface hints. Furthermore, we investigate the performance gains attributed to the Parallel Log-structured File System (PLFS) being developed by EMC Corporation and the Los Alamos National Laboratory. Our evaluation of PLFS involves two high-performance computing systems with contrasting I/O backplanes and illustrates the varied improvements to I/O that result from the deployment of PLFS (ranging from up to 25× speed-up in I/O performance on a large I/O installation to 2× speed-up on the much smaller installation at the University of Warwick).

show abstract

Section: Virtual File Systemsmentioning

confidence: 99%

Parallel File System Analysis Through Application I/O Tracing

Wright

Hammond

Pennycook

et al. 2012

The Computer Journal

View full text Add to dashboard Cite

show abstract

“…In a similar vein to [23] and [24], in which I/O throughput is vastly improved by transparently partitioning a data file (creating multiple, independent, I/O streams), PLFS uses file partitioning as well as a log-structured file system to further improve the potential I/O bandwidth. An in-depth analysis of PLFS is presented in Section 4.5.…”

Section: Related Workmentioning

confidence: 99%

Computer Performance Engineering

Thomas¹

2011

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Input/output (I/O) operations can represent a significant proportion of the run-time when large scientific applications are run in parallel. Although there have been advances in the form of file-format libraries, file system design and I/O hardware, a growing divergence exists between the performance of parallel file systems and compute processing rates.In this paper we utilise RIOT, an input/output tracing toolkit being developed at the University of Warwick, to assess the performance of three standard industry I/O benchmarks and mini-applications. We present a case study demonstrating the tracing and analysis capabilities of RIOT at scale, using MPI-IO, Parallel HDF-5 and MPI-IO augmented with the Parallel Log-structured File System (PLFS) middle-ware being developed by the Los Alamos National Laboratory.

show abstract

“…Due to their inherent portability and large scope, source-level transformations have been used for various fields within embedded systems such as code optimizations targeting I/O performance [Wang and Kaeli 2003] or energy efficiency [Chung et al 2000;Kulkarni et al 1998], formal verification [Winters and Hu 2000], and, most notably, for single-and multicore performance optimization of computationally intensive embedded applications (e.g., Falk and Marwedel [2004]; Franke and O'Boyle [2003a]; Luz and Kandemir [2004]; and Franke and O'Boyle [2003b], respectively).…”

Section: Source-to-source Transformations For Embedded Systemsmentioning

confidence: 99%

Code transformation and instruction set extension

Murray

Bennett

Franke

et al. 2009

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

The demand for flexible embedded solutions and short time-to-market has led to the development of extensible processors that allow for customization through user-defined instruction set extensions (ISEs). These are usually identified from plain C sources. In this article, we propose a combined exploration of code transformations and ISE identification. The resulting performance of such a combination has been measured on two benchmark suites. Our results demonstrate that combined code transformations and ISEs can yield average performance improvements of 49%. This outperforms ISEs when applied in isolation, and in extreme cases yields a speed-up of 2.85.

show abstract

Source level transformations to improve I/O data partitioning

Cited by 7 publications

References 15 publications

Parallel File System Analysis Through Application I/O Tracing

Parallel File System Analysis Through Application I/O Tracing

Computer Performance Engineering

Code transformation and instruction set extension

Contact Info

Product

Resources

About