Software as a service for data scientists

Abstract

F e B R uA RY 2 0 1 2 | Vo L. 5 5 | N o. 2 | c oM M u n i c aT i o n s o f T he ac M 81in performing them for terabyte or larger datasets (increasingly common across scientific disciplines) are quite different from those that applied when data volumes were measured in kilobytes. The result is a computational crisis in many laboratories and a growing need for far more powerful data-management tools, yet the typical researcher lacks the resources and expertise to operate these tools.The answer may be to deliver research data-management capabilities to users as hosted "software as a service," or SaaS, 18 a software-delivery model in which software is hosted centrally and accessed by users using a thin client (such as a Web browser) over the Internet. As demonstrated in many business and consumer tools, SaaS leverages intuitive Web 2.0 in-a S B i g D ata emerges as a force in science, 2,3 so, too, do new, onerous tasks for researchers. Data from specialized instrumentation, numerical simulations, and downstream manipulations must be collected, indexed, archived, shared, replicated, and analyzed. These tasks are not new, but the complexities involved software as a service for Data scientists The costs of research data life-cycle management are growing dramatically as data becomes larger and more complex.saas approaches are a promising solution, outsourcing time-consuming research data management tasks to third-party services.Globus online demonstrates the potential of saas for research data management, simplifying data movement for researchers and research facilities alike.

The Science DMZ: A Network Design Pattern for Data-Intensive Science

“…This packet loss was not being reported by the router's internal error monitoring, and was only noticed using the owamp active packet loss monitoring tool, which is part of the perfSONAR Toolkit. 4 Because TCP interprets the loss as network congestion, it reacts by rapidly reducing the overall sending rate. The sending rate then slowly recovers due to the dynamic behavior of the control algorithms.…”

The Sloan Digital Sky Survey Data Transfer Infrastructure

“…The SDSS collaboration has been making increasing use of Globus Online (Foster 2011;Allen et al 2012; https://www .globus.org), a general purpose bulk data transfer system that grew out of the GridFTP infrastructure. This system provides reliable bulk transfer between "endpoints" which are GridFTP servers that have advertised themselves to Globus Online.…”

Tuning GridFTP Pipelining, Concurrency and Parallelism Based on Historical Data

“…Pipelining speeds up transfers containing a large number of files, which are sent back-toback without waiting for the complete command's response for the previous transfer; this can reduce latency. As a default, pipelining (pp) is set to 4 in the previous version of GridFTP, while in Globus Online [7], pipelining is set to 5, 10 or 20, depending on the file size.…”

“…Related studies have sought to find optimal values of pipelining, concurrency and parallelism, such as Globus Online [7], which sets pipelining (pp) level to 5, 10 and 20, depending on file size. Globus Online sets parallelism (p) level to 2, 4 and 8.…”