XCPU2 distributed seamless desktop extension

Abstract

XCPU2 is the evolution of our XCPU process management system which allows the users to compose the environment of the remote cluster nodes to match that of their desktop workstation. This creates the illusion of cluster computation resources being a seamless extension of their desktop interface, facilitating cluster acceleration of workflows which can be composed and visualized on the end-user desktop. XCPU2 allows programs running on the cluster to use the same versions of the libraries and tools the user installed locally on their desktop, and access support files such as configuration and data in the familiar places in which they are located on the end-user's workstation.XCPU2 builds on our earlier work with the XCPU process management system. Like XCPU, XCPU2's interface is represented as a set of files exported by a 9P file server. It supports heterogeneous clusters and multiple head nodes. Unlike XCPU, it uses a pull instead of push model for distributed applications, binaries, and data.In this paper we describe the XCPU2 clustering model, its operation and how the per-job filesystem configuration can be used to solve some of the common problems when running a cluster.

Experiences with hybrid clusters

“…The mechanism behind creation of the session as well as initiating execution of the particular node threads is based heavily on XCPU's example [4] of using a special file, conventionally named clone, to allocate new resources. Clone files have been commonly used in Plan 9 synthetic file servers to atomically (from a file system perspective anyways) allocate and access an underlying resource.…”