The problem of staging data in workflows has received much attention over the last decade, with a variety of user-directed and automatic solutions. The latter are the focus of the first contribution in this paper, where we propose a simple peerto-peer solution adapted to the needs of RESTful services. The second contribution, is the combination of the data staging mechanism with a simple service deployment mechanism, that is designed to allow applications developed for the command-line to function as (RESTful) services without modification or (in some cases) recompilation. Thus, the aim of this paper is to describe the design and implementation of: (i) a peer-to-peer data-staging mechanism, that is itself RESTful, and (ii) a service deployment mechanism, also following REST design principles, which together form the Universal Distributed Data-flows framework, for the support of data-intensive (RESTful) workflows. We evaluate the framework by means of an engineering workflow developed for multi-disciplinary design optimization. The workflow itself is specified in Taverna, which is a conventional centralized data-staging enactment system. However, by virtue of the underlying services and staging mechanisms described here, the resulting enactment is peer-to-peer (for data), which furthermore permits asynchronous staging, with potential benefits for network utilization and end-to-end execution time.
Modern engineering is demanding a greater integration of multidisciplinary approaches. The framework for multidisciplinary optimization design turns to adopt web-based technology to construct a diversified and distributed environment for engineering computation. With the development of Internet technology, new technology can be applied and new architecture can be designed to achieve new requirements. In this paper, we present a Resource Oriented Architecture (ROA) based framework to provide a computational environment for integration of discipline specialist tools and software for MDO. The computational framework has the ability to provide a familiar and easy interface. We intend to enable the incorporation of existing knowledge residing in legacy codes, along with the access to the latest software as well as a mechanism to deal with the data-intensive nature of MDO.
A significant trend in science research for at least the past decade has been the increasing uptake of computational techniques (modelling) for insilico experimentation, which is trickling down from the grand challenges that require capability computing to smaller-scale problems suited to capacity computing. Such virtual experiments also establish an opportunity for collaboration at a distance. At the same time, the development of web service and cloud technology, is providing a potential platform to support these activities. The problem on which we focus is the technical hurdles for users without detailed knowledge of such mechanisms-in a word, 'accessibility'-specifically: (i) the heavy weight and diversity of infrastructures that inhibits shareability and collaboration between services, (ii) the relatively complicated processes associated with deployment and management of web services for non-disciplinary specialists, and (iii) the relative technical difficulty in packaging the legacy software that encapsulates key discipline knowledge for web-service environments. In this paper, we describe a lightweight framework based on cloud and REST to address the above issues. The framework provides a model that allows users to deploy REST services from the desktop on to computing infrastructure without modification or recompilation, utilizing legacy applications developed for the command-line. A behind-the-scenes facility provides asynchronous distributed staging of data (built directly on HTTP and REST). We describe the framework, comprising the service factory, data staging services and the desktop file manager overlay for service deployment, and present experimental results regarding: (i) the improvement in turnaround time from the data staging service, and (ii) the evaluation of usefulness and usability of the framework through case studies in image processing and in multidisciplinary optimization.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.