Storage, Mutability and Naming in Pasta

Moreton, Tim; Pratt, Ian; Harris, Timothy L.

doi:10.1007/3-540-45745-3_19

Cited by 10 publications

(5 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast to fingerdiff, 3DNBS adopts an index cache on the server side and can efficiently eliminate duplicate chunks within / across backup jobs in its backup phase I. In addition to the above systems, Pastiche [18] also uses CDC to identify backup buddies in a peer-to-peer system, and Pasta [19], another peer-to-peer system, also adopted this technique to save storage cost.…”

Section: Related Workmentioning

confidence: 99%

3DNBS: A Data De-duplication Disk-Based Network Backup System

Yang

Feng

Liu

et al. 2009

2009 IEEE International Conference on Networking, Architecture, and Storage

View full text Add to dashboard Cite

Traditionally, backup and archiving have been performed on tapes. With the rapid advances in disk storage technology witnessed in recent years, it becomes practical to use disks other than tape libraries as backend storage device for a backup system. For such a disk-based system, storage space efficiency is essential. Since traditional backup method cannot eliminate redundancies during backup, a new data deduplication backup technique should be developed to provide more efficient data storage at the system. This paper describes the design and performance evaluation of a data de-duplication disk-based network backup system, called 3DNBS. 3DNBS breaks files into variable sized chunks using content-defined chunking (CDC) for the purpose of duplication detection. Chunks are indexed and addressed by hashing their content, which leads to intrinsically single instance storage. Experimental results show that in comparison with traditional backup method such as Bacula, 3DNBS presents dramatic reduction in required storage space on various workloads. By eliminating duplicated data, 3DNBS also reduces the size of data to be transmitted, hence reducing time to perform backup in a bandwidth constraint environment.

show abstract

Section: Related Workmentioning

confidence: 99%

3DNBS: A Data De-duplication Disk-Based Network Backup System

Yang

Feng

Liu

et al. 2009

2009 IEEE International Conference on Networking, Architecture, and Storage

View full text Add to dashboard Cite

show abstract

“…Since CDC divides a file based on content instead of length, more duplicates can be detected than the fixed-sized blocking which is sensitive to the shifted contents. CDC has been applied to different contexts for similarity detection or duplicates suppression including backup system [8], web application [25] and distributed file protocols [20,19]. There are also works focusing on improving chunking effectiveness of the standard CDC algorithm, including the two-threshold, two-divisor (TTTD) chunking algorithm [11,12] and fingerdiff [4].…”

Section: Related Workmentioning

confidence: 99%

DEBAR: A scalable high-performance de-duplication storage system for backup and archiving

Yang

Jiang

Feng

et al. 2010

2010 IEEE International Symposium on Parallel &Amp; Distributed Processing (IPDPS)

View full text Add to dashboard Cite

We present DEBAR, a scalable and high-performance de-duplication storage system for backup and archiving, to overcome the throughput and scalability limitations of the state-of-the-art data de-duplication schemes, including the Data Domain De-duplication File System (DDFS). DEBAR uses a two-phase de-duplication scheme (TPDS) that exploits memory cache and disk index properties to judiciously turn the notoriously random and small disk I/Os of fingerprint lookups and updates into large sequential disk I/Os, hence achieving a very high de-duplication throughput. The salient feature of this approach is that both the system backup and archiving capacity and the de-duplication performance can be dynamically and cost-effectively scaled up on demand; it hence not only significantly improves the throughput of a single de-duplication server but also is conducive to distributed implementation and thus applicable to largescale and distributed storage systems.

show abstract

“…CAPFS or the Content Addressable Parallel File System [47] is a cluster file-system that exploits CAS for bandwidth savings. Various content based chunking methods have been used in Pasta [29], Pastiche [14] and REBL [24]. Ajtai et.…”

Section: Related Workmentioning

confidence: 99%

Evaluating the usefulness of content addressable storage for high-performance data intensive applications

Nath

Urgaonkar

Sivasubramaniam

2008

Proceedings of the 17th International Symposium on High Performance Distributed Computing

View full text Add to dashboard Cite

Content Addressable Storage (CAS) is a data representation technique that operates by partitioning a given data-set into non-intersecting units called chunks and then employing techniques to efficiently recognize chunks occurring multiple times. This allows CAS to eliminate duplicate instances of such chunks, resulting in reduced storage space compared to conventional representations of data. CAS is an attractive technique for reducing the storage and network bandwidth needs of performance-sensitive, data-intensive applications in a variety of domains. These include enterprise applications, Web-based e-commerce or entertainment services and highly parallel scientific/engineering applications and simulations, to name a few.In this paper, we conduct an empirical evaluation of the benefits offered by CAS to a variety of real-world data-intensive applications. The savings offered by CAS depend crucially on (i) the nature of the data-set itself and (ii) the chunk-size that CAS employs. We investigate the impact of both these factors on disk space savings, savings in network bandwidth, and error resilience of data. We find that a chunk-size of 1 KB can provide up to 84% savings in disk space and even higher savings in network bandwidth whilst trading off error resilience and incurring 14% CAS related overheads. Drawing upon lessons learned from our study, we provide insights on (i) the choice of the chunk-size for effective space savings and (ii) the use of selective data replication to counter the loss of error resilience caused by CAS.

show abstract

Storage, Mutability and Naming in Pasta

Cited by 10 publications

References 7 publications

3DNBS: A Data De-duplication Disk-Based Network Backup System

3DNBS: A Data De-duplication Disk-Based Network Backup System

DEBAR: A scalable high-performance de-duplication storage system for backup and archiving

Evaluating the usefulness of content addressable storage for high-performance data intensive applications

Contact Info

Product

Resources

About