File-system workload on a scientific multiprocessor

Kotz, David; Nieuwejaar, Nils

doi:10.1109/88.384584

Cited by 21 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…1 While some work has been done in studying the I/O needs of parallel scienti c applications (typically by examining a small number of selected applications), the CHARISMA project is unique in recording individual read and write requests in live, multiprogramming, parallel workloads. We h a ve so far completed characterization studies on an Intel iPSC/860 at NASA's Ames Research Center 1] and on a Thinking Machines CM-5 at the National Center for Supercomputing Applications 2].…”

Section: Introductionmentioning

confidence: 99%

File-access characteristics of parallel scientific workloads

Nieuwejaar

Kotz

Purakayastha

et al. 1996

IEEE Trans. Parallel Distrib. Syst.

Self Cite

140

View full text Add to dashboard Cite

Phenomenal improvements in the computational performance of multiprocessors have not been matched by comparable gains in I/O system performance. This imbalance has resulted in I/O becoming a signi cant bottleneck for many scienti c applications. One key to overcoming this bottleneck is improving the performance of parallel le systems. The design of a high-performance parallel le system requires a comprehensive understanding of the expected workload. Unfortunately, u n til recently, no general workload studies of parallel le systems have been conducted. The goal of the CHARISMA project was to remedy this problem by c haracterizing the behavior of several production workloads, on di erent machines, at the level of individual reads and writes. The rst set of results from the CHARISMA project describe the workloads observed on an Intel iPSC/860 and a Thinking Machines CM-5. This paper is intended to compare and contrast these two w orkloads for an understanding of their essential similarities and di erences, isolating common trends and platform-dependent v ariances. Using this comparison, we are able to gain more insight i n to the general principles that should guide parallel le-system design.

show abstract

Section: Introductionmentioning

confidence: 99%

File-access characteristics of parallel scientific workloads

Nieuwejaar

Kotz

Purakayastha

et al. 1996

IEEE Trans. Parallel Distrib. Syst.

Self Cite

140

View full text Add to dashboard Cite

show abstract

“…It is frequently the case that multiple compute nodes request data that is not contiguous in a file, but that the aggregation of the requests of the compute nodes is contiguous [17]. This occurs, for example, when an application has partioned a file in a strided decomposition among its processes, or when processes are each reading from or writing to the current position of a shared offset into the file.…”

Section: Dedicated I/o Nodesmentioning

confidence: 99%

Parallel I/O subsystems in massively parallel supercomputers

Feitelson

Corbett

Baylor³

et al. 1995

IEEE Parallel Distrib. Technol., Syst. Appl.

View full text Add to dashboard Cite

Applications executing on massively parallel processors (MPPs) often require a high aggregate bandwidth of lowlatency I/O to secondary storage. In many current MPPs, this requirement has been met by supplying internal parallel I/O subsystems that serve as staging areas for data. Typically, the parallel I/O subsystem is composed of I/O nodes that are linked to the same interconnection network that connects the compute nodes. The I/O nodes each manage their own set of disks. The option of increasing the number of I/O nodes together with the number of compute nodes and with the interconnection network allows for a balanced architecture. We explore the issues motivating the selection of this architecture for secondary storage in MPPs. We survey the designs of some recent and current parallel I/O subsystems, and discuss issues of system configuration, reliability, and file systems.

show abstract

“…Table 9.4. ent processes can be finely interleaved in the file [415,516]. This is reflected in the file access modes of Table 9.4 and Figure 9.45, which typically reflect a partitioning of some data among the processors.…”

Section: Parallel I/omentioning

confidence: 99%

“…Therefore different accesses interleave with each other; in many cases, this is an interleav-ing of streams of strided access used to read or write multidimensional data structures [149,515,415,516,554]. This interleaving leads to a phenomenon known as interprocess locality, because accesses by one process allow the system to predict accesses by other processes [412].…”

Section: Parallel File Systemsmentioning

confidence: 99%

Workload Modeling for Computer Systems Performance Evaluation

Feitelson

2015

191

131

View full text Add to dashboard Cite

Reliable performance evaluations require the use of representative workloads. This is no easy task since modern computer systems and their workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.

show abstract

File-system workload on a scientific multiprocessor

Cited by 21 publications

References 11 publications

File-access characteristics of parallel scientific workloads

File-access characteristics of parallel scientific workloads

Parallel I/O subsystems in massively parallel supercomputers

Workload Modeling for Computer Systems Performance Evaluation

Contact Info

Product

Resources

About