F e B R uA RY 2 0 1 2 | Vo L. 5 5 | N o. 2 | c oM M u n i c aT i o n s o f T he ac M 81in performing them for terabyte or larger datasets (increasingly common across scientific disciplines) are quite different from those that applied when data volumes were measured in kilobytes. The result is a computational crisis in many laboratories and a growing need for far more powerful data-management tools, yet the typical researcher lacks the resources and expertise to operate these tools.The answer may be to deliver research data-management capabilities to users as hosted "software as a service," or SaaS, 18 a software-delivery model in which software is hosted centrally and accessed by users using a thin client (such as a Web browser) over the Internet. As demonstrated in many business and consumer tools, SaaS leverages intuitive Web 2.0 in-a S B i g D ata emerges as a force in science, 2,3 so, too, do new, onerous tasks for researchers. Data from specialized instrumentation, numerical simulations, and downstream manipulations must be collected, indexed, archived, shared, replicated, and analyzed. These tasks are not new, but the complexities involved software as a service for Data scientists The costs of research data life-cycle management are growing dramatically as data becomes larger and more complex.saas approaches are a promising solution, outsourcing time-consuming research data management tasks to third-party services.Globus online demonstrates the potential of saas for research data management, simplifying data movement for researchers and research facilities alike.
Abstract-In this paper, we describe the design and implementation of two mechanisms for fault-tolerance and recovery for complex scientific workflows on computational grids. We present our algorithms for over-provisioning and migration, which are our primary strategies for fault-tolerance. We consider application performance models, resource reliability models, network latency and bandwidth and queue wait times for batch-queues on compute resources for determining the correct fault-tolerance strategy. Our goal is to balance reliability and performance in the presence of soft, real-time constraints like deadlines and expected success probabilities, and to do it in a way that is transparent to scientists. We have evaluated our strategies by developing a Fault-Tolerance and Recovery (FTR) service and deploying it as a part of the Linked Environments for Atmospheric Discovery (LEAD) production infrastructure. Results from real usage scenarios in LEAD show that the failure rate of individual steps in workflows decreases from about 30% to 5% by using our fault-tolerance strategies.
Web service architectures have gained popularity in recent years within the scientific grid research community. One reason for this is that web services allow software and services from various organizations to be combined easily to provide integrated and distributed applications. However, most applications developed and used by scientific communities are not web-service-oriented, and there is a growing need to integrate them into grid applications based on service-oriented architectures. In this paper, we describe a framework that allows scientists to provide a web service interface to their existing applications as web services without having to write extra code or modify their applications in any way. In addition, application providers do not need to be experts in web services standards, such as Web Services Description Language, Web Services Addressing, Web Services Security, or secure authorization, because the framework automatically generates these details. The framework also enables users to discover these application services, interact with them, and compose scientific workflows from the convenience of a grid portal.
We review the efforts of the Open Grid Computing Environments collaboration. By adopting a generalthree-tiered architecture based on common standards for portlets and Grid Web Services, we can deliver numerous capabilities to science gateways from our diverse constituent efforts. In this paper, we discuss our support for standards-based Grid portlets using the Velocity development environment. Our Grid portlets are based on abstraction layers provided by the Java CoG kit, which hide the differences of different Grid toolkits. Sophisticated services are decoupled from the portal container using Web service strategies. We describe advance information, semantic data, collaboration, and science application services developed by our consortium.
Abstract.Grid computing is about allocating distributed collections of resources including computers, storage systems, networks and instruments to form a coherent system devoted to a "virtual organization" of users who share a common interest in solving a complex problem or building an efficient agile enterprise. Service oriented architectures have emerged as the standard way to build Grids. This paper provides a brief look at the Open Grid Service Architecture, a standard being proposed by the Global Grid Forum, which provides the foundational concepts of most Grid systems. Above this Grid foundation is a layer of applicationoriented services that are managed by workflow tools and "science gateway" portals that provide users transparent access to the applications that use the resources of a Grid. In this paper we will also describe these Gateway framework services and discuss how they relate to and use Grid services.
D ata-driven computational science is characterized by dynamic adaptation in response to external data. Applications of this type, which are often data-and I/O-intensive, run as parallel or distributed computations on high-end resources such as distributed clusters or symmetric multiprocessing machines. Ondemand weather forecasting is a canonical example of a data-driven application. Essentially, it is the automated process of invoking a forecast model run in response to the detection of a severe weather condition. The existing framework for running forecast models has drawbacks that we must overcome to bring about dynamic on-demand forecasting. For instance, the current investigation process requires a lot of human involvement, particularly in the staging and moving of the files required by the model and in the invocation of downstream tools to visualize and analyze model results. Although scripts exist to automate some of the process, a knowledgeable expert must properly configure them.The Linked Environments for Atmospheric Discovery (LEAD) project addresses the limitations of current weather forecast frameworks through a new, service-oriented architecture capable of responding to unpredicted weather events and response patterns in real time. These services are intended to support the execution of multimodel simulations of weather forecasts on demand across a distributed grid of resources while dynamically adapting resource allocation in response to the results. At the system's heart is a suite of core services that together provide the essential functionality needed to invoke and run a complex experiment with minimal human involvement. Specifically, it lets the user define an experiment workflow, execute the experiment, and store the results. (See the "Related Work in Grid Technology" sidebar for a discussion of other work in this area.)In this article, we focus on three MyLEAD services-the metadata catalog service, notification service, and workflow service-that together form the core services for managing complex experimental meteorological investigations and managing the data products used in and generated during the computational experimentation. We show how the services work together on the user's behalf, easing the technological burden on the scientists and freeing them to focus on more of the science that compels them. User interaction with the system is through the LEAD portal. We've shown the ser-
Software Component Frameworks are well known in the commercial business application world and now this technology is being explored with great interest as a way to build large-scale scientific applications on parallel computers. In the case of Grid systems, the current architectural model is based on the emerging web services framework. In this paper we describe progress that has been made on the Common Component Architecture model (CCA) and discuss its success and limitations when applied to problems in Grid computing. Our primary conclusion is that a component model fits very well with a services-oriented Grid, but the model of composition must allow for a very dynamic (both in space and in time) control of composition. We note that this adds a new dimension to conventional service workflow and it extends the "Inversion of Control" aspects of most component systems.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.