Versatility and Unix semantics in namespace unification

Abstract

Administrators often prefer to keep related sets of files in different locations or media, as it is easier to maintain them separately. Users, however, prefer to see all files in one location for convenience. One solution that accommodates both needs is virtual namespace unification-providing a merged view of several directories without physically merging them. For example, namespace unification can merge the contents of several CD-ROM images without unpacking them, merge binary directories from different packages, merge views from several file servers, and more. Namespace unification can also enable snapshotting by marking some data sources read-only and then utilizing copy-on-write for the read-only sources. For example, an OS image may be contained on a read-only CD-ROM image-and the user's configuration, data, and programs could be stored in a separate read-write directory. With copy-on-write unification, the user need not be concerned about the two disparate file systems.It is difficult to maintain Unix semantics while offering a versatile namespace unification system. Past efforts to provide such unification often compromised on the set of features provided or Unix compatibility-resulting in an incomplete solution that users could not use.We designed and implemented a versatile namespace unification system called Unionfs. Unionfs maintains Unix semantics while offering advanced namespace unification features: dynamic insertion and removal of namespaces at any point in the merged view, mixing read-only and read-write components, efficient in-kernel duplicate elimination, NFS interoperability, and more. Since releasing our Linux implementation, it has been used by thousands of users and over a dozen Linux distributions, which helped us discover and solve many practical problems.

Generalized file system dependencies

“…Conversely, Featherstitch applications could benefit from the combination of strict ordering and nonblocking writes provided by xsyncfs. Like xsyncfs, stackable module software for file systems [10,24,31,38,39,42,43] and other extensions to file system and disk interfaces [12,27] might benefit from a patch-like mechanism that represented write-before relationships and consistency requirements agnostically.…”

Blutopia: Stackable storage for cluster management

“…3) Final Remarks: A previous study has quantified the overhead that large UnionFS stacks impose to the local file system and concluded that it scales well [13]. We used Postmark and a few micro-benchmarks to perform a similar study with NFS-UnionFS, varying the stack size up to 16 layers, and found that network latency was the dominating factor.…”

“…Although the current Blutopia prototype does not have the ability to coalesce adjacent layers, implementing such a feature should not pose major challenges. Besides, a previous work on UnionFS, a stackable file system for the Linux operating system, found that it is highly scalable [13]. We leverage UnionFS to implement stackable storage at the file system level, as described in Section IV.…”

“…File systems that support the notion of namespace unification, such as UnionFS [13], can be used to implement stackable storage. In this work we leverage UnionFS not only due to its widespread use, but also because the level of abstraction it offers is well suitable for our conception of stackable storage.…”

“…Therefore, our numbers do not account for the time it takes to copy a file from a read-only layer to the personality; this is done by the file system stacking approach (using UnionFS) whenever a file not present in the personality layer is modified. Wright et al [13] conducted a detailed performance analysis of UnionFS.…”