Building a production grid in scandinavia

Abstract

Innovative middleware solutions are key to the NorduGrid testbed, which spans academic institutes and supercomputing centers throughout Scandinavia and Finland and provides continuous grid services to its users. Academic researchers in the Nordic countries participate in many common projects that process large amounts of data. To function effectively, such collaborations need a grid computing infrastructure that works across a wide area and uses distributed resources efficiently. Initial evaluations of existing grid solutions, however, showed that they failed to meet this requirement. The Globus Toolkit (www.globus.org), for example, had no resource brokering capability, and the European Union's DataGrid project (EDG; www.edg.org) did not satisfy stability and reliability demands. Other grid projects had even less functionality.In May 2001, researchers at Scandinavian and Finnish academic institutes launched the NorduGrid project (www.nordugrid.org), with the goal of building a Nordic testbed for wide-area computing and data handling. We first developed a proposal for an original architecture and implementation.1,2 We then developed middleware based on the Globus libraries and API, adding a set of completely new services such as resource brokering and monitoring. The NorduGrid middleware thus preserves Globus compatibility and permits interoperability with other Globus-based solutions. It also meets our goal of providing a lightweight, yet robust solution that is noninvasive, portable, and requires minimal intervention from system administrators. We launched our testbed in May 2002, and it has been continuously operating since August of that year, providing reliable, round-the-clock services for academic users. NorduGrid spans several countries and incorporates several national computing centers, making it one of the largest operational grids in the world. NorduGrid OverviewWe built the NorduGrid testbed using the resources of national supercomputer centers and academic institutes, and based it on the Nordic countries' academic networks. Currently, the testbed uses more than 900 CPUs in 20 clusters that range in size from 4 to 400 processors. Except for a few test clusters, the resources are not grid-dedicated. Among the active NorduGrid users are high-energy physics researchers who use the Grid daily and physics theorists who use it to perform complex computational tasks. 3,4Guiding Philosophy We planned and designed the NorduGrid architecture to satisfy the needs of both users and system administrators. These needs constitute a general philosophy:• Start with something simple that works.• Avoid single points of failure.• Give resource owners full control over their resources.• Ensure that NorduGrid software can use the existing system and, eventually, other preinstalled grid middleware, such as different Globus versions.• Leave installation details (method, operating system, configuration, and so on) up to system administrators.• Pose as few restrictions on site configuration as possible -for example, permit c...

Tracing Resource Usage over Heterogeneous Grid Platforms: A Prototype RUS Interface for DGAS

“…ATLAS [11], another international HEP experiment, relies on tools and resources provided by LCG, Nor-duGrid [12] and Grid3/OSG [13], but wraps the diverse middleware stacks with a custom tool (Windmill [14]) that provides uniformity of job submission and management.…”

A Distribution Method for Solving SAT in Grids

“…We have an implementation of the method, called SATU, for NorduGrid [17] (http: //www.nordugrid.org/), a widely distributed Scandinavian computational grid. The rest of the paper is structured as follows.…”

Fine Grained Resource Reservation in Open Grid Economies

“…The Internet Virtual Organisation (iVO) [13], can be extended to utilise, on demand, resources leased dynamically [23] from Utility Computing Providers (UCP) [12,18]. A further extension of the UCP model to include the leasing of communications services, results in Utility Computing Communications Providers (UC 2 P).…”