SUMMARYMany production Grid and e-Science infrastructures have begun to offer services to end-users during the past several years with an increasing number of scientific applications that require access to a wide variety of resources and services in multiple Grids. Therefore, the Grid Interoperation Now-Community Group of the Open Grid Forum-organizes and manages interoperation efforts among those production Grid infrastructures to reach the goal of a world-wide Grid vision on a technical level in the near future. This contribution highlights fundamental approaches of the group and discusses open standards in the context of production e-Science infrastructures.
The Large Hadron Collider (LHC) at CERN, the European Organizarion for Nuclear Research, will produce unprecedented volumes of data ,when it slam operation in 2007. To provide for irs computational needs, the LHC Computing Grid {LCG) will be deployed as a worldwide computational grid service, providing the middleware upon which the physics analysis for the LHC will be carried out. In 2003, versions of this middleware were deployed which were based on the middleware produced by the European . Data Gridproject (EDG). In 2004 the LCG-2 release, which consisted of the EDG middleware with some minor mod$-cations, was deployed for use by the LHC experiments.A series of data challenges by these experiments were the first real experiment production use of LCG. During the course of the data challenges, many issues andproblems were exposed which had not shown up in mom limited rests. The deployment, service and development teams worked closely with the experiments to understand these issues and while some of the problems were solved during the data challenges, others exposed fundamental problems with the middleware as deployed in LCG-2.One of these fundamental problems was the pe$onnance under real load of the catalog component provided by EDG, the Replica Location Service. To solve these problems a new component was designed, the LCG File Catalog (LFC). The LFC moves away from the Replica Location Service model used in previous LCG releases, towards a hierarchical filesysrem model which is more like a UNIX jlesystem. it also adds missing finctionality which was requested by the experiments. This paper presents the architecture and implementation of the LFC and evaluates ir in a series of performance tests, with up to forty million entries and one hundred requesting threads from multiple clients. The results show good scalability up to the limits of these tests, and compare favourably with other Grid catalog implementations.
No abstract
Storage management is one of the most important enabling technologies for large-scale scientific investigations.Having to deal with multiple heterogeneous storage and file systems is one of the major bottlenecks in managing, replicating, and accessing files in distributed environments. Storage Resource Managers (SRMs), named after their web services control protocol, provide the technology needed to manage the rapidly growing distributed data volumes, as a result of faster and larger computational facilities. SRMs are Grid storage services providing interfaces to storage resources, as well as advanced functionality such as dynamic space allocation and file management on shared storage systems. They call on transport services to bring files into their space transparently and provide effective sharing of files. SRMs are based on a common specification that emerged over time and evolved into an international collaboration. This approach of an open specification that can be used by various institutions to adapt to their own storage systems has proven to be a remarkable success -the challenge has been to provide a consistent homogeneous interface to the Grid, while allowing sites to have diverse infrastructures.In particular, supporting optional features while preserving interoperability is one of the main challenges we describe in this paper. We also describe using SRM in a large international High Energy Physics collaboration, called WLCG, to prepare to handle the large volume of data expected when the Large Hadron Collider (LHC) goes online at CERN. This intense collaboration led to refinements and additional functionality in the SRM specification, and the development of multiple interoperating implementations of SRM for various complex multicomponent storage systems.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.