Toward Transparent Data Management in Multi-Layer Storage Hierarchy of HPC Systems

Wadhwa, Bharti; Byna, Suren; Butt, Ali R.

doi:10.1109/ic2e.2018.00046

Cited by 11 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several state-of-the-art multi-level caching techniques were proposed in the literature [29][30][31][32] which mainly concentrates on caching the data based on prediction by analyzing the characteristics of I/O accesses. The authors of [33] predicted the life time of files by analyzing the access frequency of the files.…”

Section: Literature On Prefetching and Caching Techniquesmentioning

confidence: 99%

HRFP: Highly Relevant Frequent Patterns-Based Prefetching and Caching Algorithms for Distributed File Systems

et al. 2023

View full text Add to dashboard Cite

Data-intensive applications are generating massive amounts of data which is stored on cloud computing platforms where distributed file systems are utilized for storage at the back end. Most users of those applications deployed on cloud computing systems read data more often than they write. Hence, enhancing the performance of read operations is an important research issue. Prefetching and caching are used as important techniques in the context of distributed file systems to improve the performance of read operations. In this research, we introduced a novel highly relevant frequent patterns (HRFP)-based algorithm that prefetches content from the distributed file system environment and stores it in the client-side caches that are present in the same environment. We have also introduced a new replacement policy and an efficient migration technique for moving the patterns from the main memory caches to the caches present in the solid-state devices based on a new metric namely the relevancy of the patterns. According to the simulation results, the proposed approach outperformed other algorithms that have been suggested in the literature by a minimum of 15% and a maximum of 53%.

show abstract

Section: Literature On Prefetching and Caching Techniquesmentioning

confidence: 99%

HRFP: Highly Relevant Frequent Patterns-Based Prefetching and Caching Algorithms for Distributed File Systems

et al. 2023

View full text Add to dashboard Cite

show abstract

“…The I/O hierarchy in supercomputers clearly exhibits a trend of being more complicated due to the diverse devices and performance requirements from increasingly datadriven scientific discovery needs (Wadhwa et al 2018). HPC systems tend to use high-speed devices to fill up the gap between massive computing capacity and slow I/O subsystems.…”

Section: Characteristics Of Pre-exascale and Projected Exascale I/o Amentioning

confidence: 99%

Software-defined QoS for I/O in exascale computing

et al. 2019

View full text Add to dashboard Cite

Supercomputers' capability is approaching the exascale level, which enables large computing systems to run more jobs concurrently. Since modern data-intensive scientific applications can sometimes produce millions of I/O requests per second, I/O systems always suffer from heavy workloads and impede the overall performance. How to allocate I/O resources and guarantee the QoS (Quality of Service) for each individual application is becoming an increasingly important question. In this paper, we propose SDQoS, a software-defined QoS framework with the token bucket algorithm, aiming to meet the I/O requirements of concurrent applications contending for the I/O resources and improve the overall performance of the I/O systems. Evaluation shows that SDQoS can effectively control the I/O bandwidth within a 5%-10% deviation and improve the performance by 20% in extreme cases.

show abstract

“…Highly configurable large-scale distributed storage systems require high hardware investment, and the cost of hardware devices has become a stumbling block for research work [11,12]. At the same time, the performance of block storage is greatly affected by hardware configurations, and the performance metrics obtained from different combinations of hardware configurations vary greatly [13][14][15][16].…”

Section: Introductionmentioning

confidence: 99%

A virtualisation performance evaluation approach for distributed block device

2023

IJFET

View full text Add to dashboard Cite

With the development of the information age, the volume of data is exploding and the operational and performance requirements of massive data are difficult to balance. The high cost of distributed storage systems with high data reliability and excellent random read and write performance, often consisting of multiple racks, multiple nodes and hundreds of disks, is a thorny issue for researchers. A tool to predict the performance of distributed storage architectures has been built based on a study of the cost of distributed storage architectures. Without hardware, enabling low cost and fast performance prediction. It also provides performance data and latency density to compare the performance of different workloads and architectures more intuitively, providing ideas for optimising the performance of distributed systems.

show abstract

Toward Transparent Data Management in Multi-Layer Storage Hierarchy of HPC Systems

Cited by 11 publications

References 14 publications

HRFP: Highly Relevant Frequent Patterns-Based Prefetching and Caching Algorithms for Distributed File Systems

HRFP: Highly Relevant Frequent Patterns-Based Prefetching and Caching Algorithms for Distributed File Systems

Software-defined QoS for I/O in exascale computing

A virtualisation performance evaluation approach for distributed block device

Contact Info

Product

Resources

About