High-performance data management for genome sequencing centers using Globus Online: A case study

Abstract

In the past few years in the biomedical field, availability of low-cost sequencing methods in the form of nextgeneration sequencing has revolutionized the approaches life science researchers are undertaking in order to gain a better understanding of the causative factors of diseases. With biomedical researchers getting many of their patients' DNA and RNA sequenced, sequencing centers are working with hundreds of researchers with terabytes to petabytes of data for each researcher. The unprecedented scale at which genomic sequence data is generated today by high-throughput technologies requires sophisticated and high-performance methods of data handling and management. For the most part, however, the state of the art is to use hard disks to ship the data. As data volumes reach tens or even hundreds of terabytes, such approaches become increasingly impractical. Data stored on portable media can be easily lost, and typically is not readily accessible to all members of the collaboration. In this paper, we discuss the application of Globus Online within a sequencing facility to address the data movement and management challenges that arise as a result of exponentially increasing amount of data being generated by a rapidly growing number of research groups. We also present the unique challenges in applying a Globus Online solution in sequencing center environments and how we overcome those challenges.

Experiences building Globus Genomics: a next‐generation sequencing analysis service using Galaxy, Globus, and Amazon Web Services

“…Before adopting Globus Genomics, raw sequencing data of tens of terabytes were shipped on hard disks using services such as FedEx from the sequencing facilities to the Dobyns lab. Dr. Dobyns and his two postdocs would then analyze the data on a single multicore machine within their laboratory.…”