Distributed data storage for opportunistic grids

Abstract

Grid applications typically need to deal with large amounts of data. The traditional approach for data storage is to employ high-performance dedicated servers with data replication. However, a class of computational grids, called opportunistic grids, focus on the usage of idle resources from shared machines. These machines normally have large quantities of unused storage space that could be used when the machines are idle, allowing opportunistic grids to share not only computational cycles, but also storage space.In this work, we present the initial design of OppStore, a middleware that provides reliable storage using the free storage space from shared grid machines. The storage can be transparently accessed from any grid machine, allowing easy data sharing among grid users and applications. The system uses a two-level peerto-peer organization to connect grid machines in a scalable and fault-tolerant way. To deal with resource heterogeneity, we developed the concept of virtual ids, which allows the creation of virtual spaces located on top of the peer-to-peer routing substrate. These virtual spaces enables the middleware to perform heterogeneityaware, load-balancing selection of storage sites using multiple simultaneous metrics.

Reliable management of checkpointing and application data in opportunistic grids

“…To deal with this limitation, we assign to each node an additional identifier, called virtual id [9], using the same range of valid identifiers from Pastry, which we now call virtual id space. In our system, each CDRM receives a virtual id and becomes responsible for a virtual id range proportional to the capacity of the cluster.…”

Design and Implementation of a Middleware for Data Storage in Opportunistic Grids

“…To deal with these limitations, we assign to each node an additional identifier, called virtual id [7], using the same range of valid identifiers from Pastry, which we now call virtual id space. Virtual ids can be changed dynamically to reflect the heterogeneity of nodes and to adapt to dynamic environments where nodes constantly leave and join the network or change their characteristics.…”

“…In a previous work [7], we proposed OppStore, a middleware that explores the usage of a peer-to-peer overlay network for storage of read-only data in non-dedicated machines in the context of opportunistic grids. We also introduced the concept of virtual ids to deal with load-balancing and node heterogeneity.…”