This paper addresses the inherent unreliability and instability of worker nodes in large-scale donationbased distributed infrastructures such as P2P and Grid systems. We present adaptive scheduling techniques that can mitigate this uncertainty and significantly outperform current approaches. In this work, we consider nodes that execute tasks via donated computational resources and may behave erratically or maliciously. We present a model in which reliability is not a binary property but a statistical one based on a node's prior performance and behavior. We use this model to construct several reputationbased scheduling algorithms that employ estimated reliability ratings of worker nodes for efficient task allocation. Our scheduling algorithms are designed to adapt to changing system conditions as well as non-stationary node reliability. Through simulation we demonstrate that our algorithms can significantly improve throughput, while maintaining a very high success rate of task completion. Our results suggest that reputation-based scheduling can handle wide variety of worker populations, including non-stationary behavior, with overhead that scales well with system size. We also show that our adaptation mechanism allows the application designer fine-grain control over desired performance metrics.

Virtualization is being widely used in large-scale computing environments, such as clouds, data centers, and grids, to provide application portability and facilitate resource multiplexing while retaining application isolation. In many existing virtualized platforms, it has been found that the network bandwidth often becomes the bottleneck resource, causing both high network contention and reduced performance for communication and data-intensive applications. In this paper, we present a decentralized affinity-aware migration technique that incorporates heterogeneity and dynamism in network topology and job communication patterns to allocate virtual machines on the available physical resources. Our technique monitors network affinity between pairs of VMs and uses a distributed bartering algorithm, coupled with migration, to dynamically adjust VM placement such that communication overhead is minimized. Our experimental results running the Intel MPI benchmark and a scientific application on a 7-node Xen cluster show that we can get up to 42% improvement in the runtime of the application over a no-migration technique, while achieving up to 85% reduction in network communication cost. In addition, our technique is able to adjust to dynamic variations in communication patterns and provides both good performance and low network contention with minimal overhead. We also present a topology-aware extension to our migration algorithm that provides an additional 26-31% reduction in runtime.

Abstract

Reputation systems have been a hot topic in the peer-to-peer community for several years. In a services-oriented distributed computing environment like the Grid, reputation systems can be utilized by clients to select between competing service providers. In this paper, we selected several existing reputation algorithms and adapted them to the problem of service selection in a Grid-like environment. We performed a quantitative comparison of both the accuracy and overhead associated with these techniques under common scenarios. The results indicate that using a reputation system to guide service selection can significantly improve client satisfaction with minimal overhead. In addition, we show that the most appropriate algorithm depends on the kinds of anticipated attacks. A new algorithm we've proposed appears to be the approach of choice if clients can misreport service ratings.