Optimizing Local File Accesses for FUSE-Based Distributed Storage

Ishiguro, Shun; Murakami, Jun; Oyama, Yoshihiro; Tatebe, Osamu

doi:10.1109/sc.companion.2012.104

Cited by 11 publications

(11 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve FUSE performance, Narayan et al proposed to marry in-kernel stackable file systems [61] with FUSE [39]. Shun et al modified FUSE's kernel module to allow applications to access storage devices directly [30]. These improvements were in research prototypes and were never included in the mainline.…”

Section: Related Workmentioning

confidence: 99%

Performance and Resource Utilization of FUSE User-Space File Systems

Vangoor¹,

Agarwal

Mathew

et al. 2019

ACM Trans. Storage

View full text Add to dashboard Cite

Traditionally, file systems were implemented as part of operating systems kernels, which provide a limited set of tools and facilities to a programmer. As the complexity of file systems grew, many new file systems began being developed in user space. Low performance is considered the main disadvantage of user-space file systems but the extent of this problem has never been explored systematically. As a result, the topic of user-space file systems remains rather controversial: while some consider user-space file systems a "toy" not to be used in production, others develop full-fledged production file systems in user space. In this article, we analyze the design and implementation of a well-known user-space file system framework, FUSE, for Linux. We characterize its performance and resource utilization for a wide range of workloads. We present FUSE performance and also resource utilization with various mount and configuration options, using 45 different workloads that were generated using Filebench on two different hardware configurations. We instrumented FUSE to extract useful statistics and traces, which helped us analyze its performance bottlenecks and present our analysis results. Our experiments indicate that depending on the workload and hardware used, performance degradation (throughput) caused by FUSE can be completely imperceptible or as high as −83%, even when optimized; and latencies of FUSE file system operations can be increased from none to 4× when compared to Ext4. On the resource utilization side, FUSE can increase relative CPU utilization by up to 31% and underutilize disk bandwidth by as much as −80% compared to Ext4, though for many data-intensive workloads the impact was statistically indistinguishable. Our conclusion is that user-space file systems can indeed be used in production (non-"toy") settings, but their applicability depends on the expected workloads.

show abstract

Section: Related Workmentioning

confidence: 99%

Performance and Resource Utilization of FUSE User-Space File Systems

Vangoor¹,

Agarwal

Mathew

et al. 2019

ACM Trans. Storage

View full text Add to dashboard Cite

show abstract

“…Zhang et.al [11] composite the small files together to improve the file system performance under metadata-centric workload. Shun Ishiguro et al [33] optimized local file accesses for FUSE-based distributed storage. Yan et al [12] reduce the kernel/user mode switches in the optical library file system.…”

Section: Related Workmentioning

confidence: 99%

Defuse: Decoupling Metadata and Data Processing in FUSE Framework for Performance Improvement

2019

View full text Add to dashboard Cite

A popular user-space file system framework, FUSE, has been widely used for building various customized file systems(cFS) on top of the underlying kernel file system(kFS). A FUSE-based cFS gains adequate flexibility by developing its specific functions in user space, but brings extra user-kernel mode switches in the request processing flow owing to forwarding all requests from the FUSE kernel driver to the user-space daemon, thus degrading the overall performance. We observe that a file data request does not need to forward to the user-space daemon when its file-to-file mapping between the FUSE-based cFS and the underlying kFS remains unchanged. Based on the insight, we propose a modified FUSE framework -DeFUSE that decouples the processing flow of the metadata and data requests. The metadata requests still follow the original flow to reserve flexibility while the data requests are directly executed in the DeFUSE kernel driver maintaining the file-to-file mappings in the kernel, effectively eliminating the unnecessary mode switches. We have implemented the DeFUSE framework and ported three representative FUSE-based cFSs to DeFUSE. The result shows that for data-centric workloads, the throughput of DeFUSE-based cFSs increases up to 3.5X for write and 3.8X for read respectively, compared to their corresponding FUSE-based implementations. DeFUSE is available on Github.

show abstract

“…Also, Exa-Scale Storage contains various storage devices or networks due to advances in technology [2,3]. In this complex environment, the management costs will vary significantly depending on how file management is performed in the ExaScale environment, when the user performs the application, it varies the processing time and processing costs according to the processing method [4][5][6][7].…”

Section: Introductionmentioning

confidence: 99%

Remote Direct Storage Management for Exa-Scale Storage

Kim¹,

Cha²,

Kim³

2016

Advanced Science and Technology Letters

View full text Add to dashboard Cite

Abstract.Recently, the size of storage has been increasing in order to store large amounts of data. The most part of storage research is focused on raising capacity and bandwidth. But, efficiency of the file management is becoming even more important in terms of in Exa-Scale Storage operations. In this paper, we present a method of Remote Direct Storage Management (RDSM) for ExaScale Storage. RDSM allows users to easily manage server-side file and to easily use storage-specific functions. By utilizing RDSM, file copy is up to 30% faster than cp and file movement is up to 240 times faster than mv in LINUX.

show abstract

Optimizing Local File Accesses for FUSE-Based Distributed Storage

Cited by 11 publications

References 7 publications

Performance and Resource Utilization of FUSE User-Space File Systems

Performance and Resource Utilization of FUSE User-Space File Systems

Defuse: Decoupling Metadata and Data Processing in FUSE Framework for Performance Improvement

Remote Direct Storage Management for Exa-Scale Storage

Contact Info

Product

Resources

About