Evaluating the usefulness of content addressable storage for high-performance data intensive applications

Abstract

Content Addressable Storage (CAS) is a data representation technique that operates by partitioning a given data-set into non-intersecting units called chunks and then employing techniques to efficiently recognize chunks occurring multiple times. This allows CAS to eliminate duplicate instances of such chunks, resulting in reduced storage space compared to conventional representations of data. CAS is an attractive technique for reducing the storage and network bandwidth needs of performance-sensitive, data-intensive applications in a variety of domains. These include enterprise applications, Web-based e-commerce or entertainment services and highly parallel scientific/engineering applications and simulations, to name a few.In this paper, we conduct an empirical evaluation of the benefits offered by CAS to a variety of real-world data-intensive applications. The savings offered by CAS depend crucially on (i) the nature of the data-set itself and (ii) the chunk-size that CAS employs. We investigate the impact of both these factors on disk space savings, savings in network bandwidth, and error resilience of data. We find that a chunk-size of 1 KB can provide up to 84% savings in disk space and even higher savings in network bandwidth whilst trading off error resilience and incurring 14% CAS related overheads. Drawing upon lessons learned from our study, we provide insights on (i) the choice of the chunk-size for effective space savings and (ii) the use of selective data replication to counter the loss of error resilience caused by CAS.

Active Data Structures on GPGPUs

“…These comparisons are performed in parallel; the time to search n items is typically reduced by a factor of at least O(log n). Associative memory can reduce communication costs for data intensive applications, with organisation of the data into suitable chunks [10].…”

Untitled

“…Cloud storage systems for the most part contain a lot of information that basically require quickly and exact information recovery to support intelligent and adaptive cloud administrations [1][2] [3]. For instance, 7 percent of shoppers put away their substance in the cloud in 2011, and the figure will develop to 36 percent in 2016, as per the Gartner, Inc. [4] and Storage Newsletter [5] reports.…”

Real-Time Semantic Search Using Approximate Methodology for Large-Scale Storage Systems

“…Cloud storage systems generally contain large amounts of data that critically require fast and accurate data retrieval to support intelligent and adaptive cloud services [1]- [3]. For example, 7% of consumers stored their contents in the cloud in 2011, and the figure will grow to 36% in 2016, according to the Gartner, Inc. [4] and Storage Newsletter [5] reports.…”