Abstract-Emerging scientific simulations on leadership class systems are generating huge amounts of data. However, the increasing gap between computation and disk IO speeds makes traditional data analytics pipelines based on post-processing cost prohibitive and often infeasible. In this paper, we investigate an alternate approach that aims to bring the analytics closer to the data using data staging and the in-situ execution of data analysis operations. Specifically, we present the design, implementation and evaluation of a framework that can support in-situ featurebased object tracking on distributed scientific datasets. Central to this framework is the scalable decentralized and online clustering (DOC) and cluster tracking algorithm, which executes in-situ (on different cores) and in parallel with the simulation processes, and retrieves data from the simulations directly via on-chip shared memory. The results from our experimental evaluation demonstrate that the in-situ approach significantly reduces the cost of data movement, that the presented framework can support scalable feature-based object tracking, and that it can be effectively used for in-situ analytics for large scale simulations.
Emerging scientific simulations on leadership class systems are generating huge amounts of data and processing this data in an efficient and timely manner is critical for generating insights from the simulations. However, the increasing gap between computation and disk I/O speeds makes traditional data analytics pipelines based on post-processing cost prohibitive and often infeasible. In this paper, we investigate an alternate approach that aims to bring the analytics closer to the data using in-situ execution of data analysis operations. Specifically, we present the design, implementation and evaluation of a framework that can support in-situ feature-based objects tracking on distributed scientific datasets. Central to this framework is a scalable decentralized and online clustering, a cluster tracking algorithm, which executes in-situ (on different cores) in parallel with the simulation processes, and retrieves data from the simulations directly via on-chip shared memory. The results from our experimental evaluation demonstrate that the in-situ approach significantly reduces the cost of data movement, that the presented framework can support scalable featurebased objects tracking, and that it can be effectively used for in-situ analytics in large scale simulations.
The purpose of this chapter is to identify and analyze the challenges of creating new software in the public cloud due to legal regulations. Specifically, this chapter explores how the Sarbanes-Oxley Act (SOX) will indirectly affect the development and implementation process of cloud computing applications in terms of software engineering and actual legality of said software solutions. The goal of this chapter is twofold - to bring attention to the need for specific analysis of legal issues in public clouds (as opposed to general analysis), and to illustrate the need for cloud developers to address legal constraint while creating their platforms, in order to increase their viability in the corporate environment.
The purpose of this chapter is to identify and analyze the challenges of creating new software in the public cloud due to legal regulations. Specifically, this chapter explores how the Sarbanes-Oxley Act (SOX) will indirectly affect the development and implementation process of cloud computing applications in terms of software engineering and actual legality of said software solutions. The goal of this chapter is twofold - to bring attention to the need for specific analysis of legal issues in public clouds (as opposed to general analysis), and to illustrate the need for cloud developers to address legal constraint while creating their platforms, in order to increase their viability in the corporate environment.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.