Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Desktop Workstations

Abstract

Client-side data caching serves as an excellent mechanism to store and analyze the rapidly growing scientific data, motivating distributed, client-side caches built from unreliable desktop storage contributions to store and access large scientific data. They offer several desirable properties, such as performance impedance matching, improved space utilization, and high parallel I/O bandwidth. In this context, we are faced with two key challenges: (1) the finite amount of contributed cache space is stretched This work builds on several of our previously published studies. While we included extended introduction of the FreeLoader framework [57,58] as well as our combination of prefix caching and collective download techniques [33], this paper proposes the novel RDPR technique as well as a new striping-enabled cache management algorithm. We also discussed methods to combine RDPR with prefix caching and collective download techniques. Overall, more than 60% of the manuscript's content has not been previously published.X. Ma · Z. Zhang (B) North Carolina State University, Raleigh, NC, USA e-mail: zzhang3@ncsu.edu X. Ma · S. S. Vazhkudai Oak Ridge National Laboratory, Oak Ridge, TN, USA by the ever increasing scientific dataset sizes and (2) the transient nature of volunteered storage nodes impacts data availability. In this article, we address these challenges by exploiting the existence of external, primary copies of datasets. We propose a novel combination of prefix caching, collective download, and remote partial data recovery (RPDR), to deal with optimal cache space consumption and storage node volatility. Our evaluation, performed on our FreeLoader prototype, indicates that prefix caching can significantly improve the cache hit rate and partial data recovery is better than (or comparable to) many persistent-data availability techniques.

WiGriMMA: A Wireless Grid Monitoring Model Using Agents

“…To achieve guaranteed performance the proposal in [24] presents an adaptive execution scheme using agents, so that an application can adapt to its run-time environment based on the resource availability and application demands. The problem of optimal cache space consumption and storage node volatility to improve the data availability for better access performance is addressed in [25]. Considering user mobility and resource availability the works presented in [26] and [27] present scheduling algorithms in mobile Grid to provide reliable performance.…”