SZTAKI Desktop Grid: a Modular and Scalable Way of Building Large Computing Grids

Abstract

So far BOINC based desktop Grid systems have been applied at the global computing level. This paper describes an extended version of BOINC called SZTAKI Desktop Grid (SZDG) that aims at using Desktop Grids (DGs) at local (enterprise/institution) level. The novelty of SZDG is that it enables the hierarchical organisation of local DGs, i.e., clients of a DG can be DGs at a lower level that can take work units from their higher level DG server. More than that, even clusters can be connected at the client level and hence work units can contain complete MPI programs to be run on the client clusters. In order to easily create Master/Worker type DG applications a new API, called as the DC-API has been developed. SZDG and DC-API has been successfully applied both at the global and local level, both in academic institutions and in companies to solve problems requiring large computing power.

Evaluating the performance and intrusiveness of virtual machines for desktop grid computing

“…Their main goal is to devise a generic architecture that can be transparently integrated with desktop grid middleware (in particular, the Sztaki DG and extensions are targeted [4]). The proposed architecture relies on a virtual machine base image, which is used to create instances.…”

“…The choice was motivated by the flexibility of the virtual environment, when compared to other system level virtual machines. Indeed, QEMU provides overlay and copy-on-write images, backdoor access and single user process instance [4]. For Csaba et al, flexibility-related issues have higher precedence than performance, since deployment and manageability are determinant factors when considering virtual environments for desktop grid computing.…”

EDGeS: A Bridge between Desktop Grids and Service Grids

“…Resources of volunteer DGs are collected from individual desktop owners as their volunteer contribution to the Desktop Grid. Typical public, volunteer DGs are the BOINC-based DG systems like SETI@HOME, Einstein@HOME [5], SZTAKI Desktop Grid [6], etc. AlmereGrid [7] and XtremWeb [8] are also volunteer, public DG systems.…”

An implementation of parallel file distribution in an agent hierarchy

“…The AgentTeamwork system delivers files to each remote process through four distinctive steps: (1) partitioning files into stripes, (2) transferring them through an agent hierarchy, (3) check-pointing them for recovery purposes, and (4) maintaining file-stripe consistency among different processes. The following subsections describe details of each implementation.…”

“…PC grid [1,2] is such a cost-effective platform that attracts users by collecting clusters of research, instructional, and even office desktops. However, for dataintensive applications, PC grid needs to pay more attention to file availability than other platforms.…”