Taming aggressive replication in the Pangaea wide-area file system

Abstract

Pangaea is a wide-area file system that supports data sharing among a community of widely distributed users. It is built on a symmetrically decentralized infrastructure that consists of commodity computers provided by the end users. Computers act autonomously to serve data to their local users. When possible, they exchange data with nearby peers to improve the system's overall performance, availability, and network economy. This approach is realized by aggressively creating a replica of a file whenever and wherever it is accessed.This paper presents the design, implementation, and evaluation of the Pangaea file system. Pangaea offers efficient, randomized algorithms to manage highly dynamic and potentially large groups of file replicas. It applies optimistic consistency semantics to replica contents, but it also offers stronger guarantees when required by the users. The evaluation demonstrates that Pangaea outperforms existing distributed file systems in large heterogeneous environments, typical of the Internet and of large corporate intranets.

VisageFS Translucent Directories: Efficient Posix Consistencies for Wide-Area Workflows

“…The optimistic replication concept [28] used in large-scale file systems like Coda [27], Pangaea [4] and Ficus [29] leads to costly reconciliation protocols. The Ficus reconciliation protocol makes use of a combination of version vectors [30] and ghost entries to detect creation / deletion conflicts.…”

The design of novel distributed protocols from differential equations

“…Liveness ensures that a responsible process deletes the file eventually. 4 Partial data replication has been studied for many years, e.g., in databases [11,13], email and file systems [16,24,25], and more recently, in peer to peer systems [7]. Gray et al [13] argue against the dangers of large-scale replication and warn that maintaining consistency among a large number of replicas might counter the goal of scalability.…”

“…Most existing solutions to replica location [23,24] locate replicas statically and reactively, i.e., the subset of hosts selected to hold replicas of a given object does not change unless a special event happens, e.g., one of the hosts could crash. This approach has the following three disadvantages: (1) it can be expensive in systems containing millions of hosts, where a large fraction have short lifetimes [O(several minutes)] and rejoin multiple times (6.4 times/day as reported in the Overnet system [5]); 5 (2) From a security stand-point, static and reactive strategies allow an attacker to easily locate and attack all the individual replicas of an object.…”

Conflict Resolution for User-Selected Names in Collaborative Systems

“…for U2) then become the previous names, and ni becomes the new current name for both users. This mechanism is similar to the back pointers introduced by Saito et al in the Pangea filesystem [19] where it is used for consistency. We use back pointers to improve usability, instead.…”