‘Big data’, Hadoop and cloud computing in genomics

Abstract

Since the completion of the Human Genome project at the turn of the Century, there has been an unprecedented proliferation of genomic sequence data. A consequence of this is that the medical discoveries of the future will largely depend on our ability to process and analyse large genomic data sets, which continue to expand as the cost of sequencing decreases. Herein, we provide an overview of cloud computing and big data technologies, and discuss how such expertise can be used to deal with biology's big data sets. In particular, big data technologies such as the Apache Hadoop project, which provides distributed and parallelised data processing and analysis of petabyte (PB) scale data sets will be discussed, together with an overview of the current usage of Hadoop within the bioinformatics community.

Biomarker discovery, high performance and cloud computing: A comprehensive review

“…Clouds have been widely used in bioinformatics [47], [48], [49], [50], some of which have already been mentioned above. Cloud computing provides resources, data storage and processing as a service on a "pay as you go" basis, using a Service Level Agreement (SLA) [51] protocol.…”

Research of Big Data Based on the Views of Technology and Application

“…According to the McKinsey Global Institute, effective use of big data will help the US health care sector save $300 billion per year in savings, reduce spending by 8% [14]. Big Data analysis can be applied to echocardiography, angiography, and magnetic resonance imaging or computed tomography to form cardiac imaging [15].…”

Semantic Web technologies for the big data in life sciences

“…From 2005, the sequencing output doubling rate decreased to 5 months because of the development of Next-Generation Sequencing technologies (NGS) (5). Since 2008, genomics data are outpacing Moore's Law by a factor of 4 (6). The 1,000 Genomes Project (7), which involves sequencing and cataloging human genetic variations, has deposited 2 times more raw data into GenBank (http://www.ncbi.nlm.…”

“…Figure 1 illustrates the GenBank and Whole Genome Shotgun (WGS) statistics up to February 2014. Human DNA comprises approximately 3 billion base pairs with a personal genome representing approximately 100 gigabyte (GB) of data (6). Two nanopore sequencing platforms (GridION TM and MinION TM ) (9), reported in February 2012, are capable of delivering ultra-long sequencing reads (~100 kb) with additionally higher throughput and much lower cost.…”