Evaluating cloud computing in the NASA DESDynI ground data system

Abstract

The proposed NASA Deformation, Ecosystem Structure and Dynamics of Ice (DESDynI) mission would be a first-ofbreed endeavor that would fundamentally change the paradigm by which Earth Science data systems at NASA are built. DESDynI is evaluating a distributed architecture where expert science nodes around the country all engage in some form of mission processing and data archiving. This is compared to the traditional NASA Earth Science missions where the science processing is typically centralized. What's more, DESDynI is poised to profoundly increase the amount of data collection and processing well into the 5 terabyte/day and tens of thousands of job range, both of which comprise a tremendous challenge to DESDynI's proposed distributed data system architecture. In this paper, we report on a set of architectural trade studies and benchmarks meant to inform the DESDynI mission and the broader community of the impacts of these unprecedented requirements. In particular, we evaluate the benefits of cloud computing and its integration with our existing NASA ground data system software called Apache Object Oriented Data Technology (OODT). The preliminary conclusions of our study suggest that the use of the cloud and OODT together synergistically form an effective, efficient and extensible combination that could meet the challenges of NASA science missions requiring DESDynI-like data collection and processing volumes at reduced costs.

Architecting Scientific Data Systems in the Cloud

“…Sev eral of the case studies presented in this chapter were fi rst described at the workshop SECLOUD , or Software Engineering for Cloud Computing, in particular, the Lunar Mapping and Modeling (LMMP) project use case [ 14 ], and the DESDynI science data processing system use case [ 15 ].…”

Green Streams for data-intensive software

“…From megabyte-scale Youtube Apps on smart phones, to gigabyte-scale Netflix applications on laptops, and to terabytescale NASA scientific computations on servers [1], dataintensive software is quickly becoming the new norm of modern computing. Software engineering for "Big Data" is an active area of research, with innovations addressing diverse goals, such as architectural soundness [2], [3], programmability [4], [5], performance [6], [7], and seamless database integration [8], [9].…”