Beyond striping: the bridge multiprocessor file system

Abstract

High-performance parallel computers require high-performance file systems. Exotic I/O hardware will be of little use if file system software runs on a. single processor of a many-processor machine. We believe that cost-effective I/O for large multiprocessors can best be obtained by spreading both data and file system computation over a large number of processors and disks. To assess the effectiveness of this approach, we have implemented a prototype system called Bridge, and have studied its performance on several data intensive applications, among them external sorting. A detailed analysis of our sorting algorithm indicates that Bridge can profitably be used on configurations in excess of one hundred processors with disks. Empirical results on a 32-processor implementation agree with the analysis, providing us with a high degree of confidence in this prediction. Based on our experience, we argue that file systems such as Bridge will satisfy the I/O needs of a wide range of parallel architectures and applications.

A brief survey of current work on network attached peripherals (extended abstract)

“…Hyer et all 33] discuss the hazards of getting a NAP to cooperate with an existing system, pointing out aws in IPI-3 that make it unsuitable for third-party use, especially the lack of an authentication mechanism. Using the IPI-3 COPY command results in the mover 20 for one of the devices being left out of the command loop, making device management more complex. Their solution involves creating a variant of IPI-3 third party transfer in which the initiating device itself sends a request to the responder's mover rather than directly from the device itself.…”

“…Also relevant is the work on operating systems for distributed-memory multicomputers 63], such as the Intel Touchstone Delta/Paragon 34 family, which has some nodes dedicated to I/O and others to computation, and the IBM SP and Cray T3D machines. Numerous studies on I/O performance 10] and le system design 46,20,17,19] have been done. Some of this work includes, for example, distributed le block layout and synchronization mechanisms that may prove useful for NAP le systems.…”

Serverless network file systems

“…Several file systems, such as CFS [l?ier89], Bridge[Dibb89], and Vesta[Corb93], distribute clata over multiple storage servers to support parallel workloads; however, theyOther parallel systems have imp]ementeci redundant data storage intended for restricted workloads consisting entirely of large files, where per-file striping is appropriate and where large file accesses reduce stress on their centralized manager architectures. For instance, Swift [Cabrera9 lb] and SFS [LoVe93] provide redundant distributed data storage for…”