Benchmarking Undedicated Cloud Computing Providers for Analysis of Genomic Datasets

Abstract

A major bottleneck in biological discovery is now emerging at the computational level. Cloud computing offers a dynamic means whereby small and medium-sized laboratories can rapidly adjust their computational capacity. We benchmarked two established cloud computing services, Amazon Web Services Elastic MapReduce (EMR) on Amazon EC2 instances and Google Compute Engine (GCE), using publicly available genomic datasets (E.coli CC102 strain and a Han Chinese male genome) and a standard bioinformatic pipeline on a Hadoop-based platform. Wall-clock time for complete assembly differed by 52.9% (95% CI: 27.5–78.2) for E.coli and 53.5% (95% CI: 34.4–72.6) for human genome, with GCE being more efficient than EMR. The cost of running this experiment on EMR and GCE differed significantly, with the costs on EMR being 257.3% (95% CI: 211.5–303.1) and 173.9% (95% CI: 134.6–213.1) more expensive for E.coli and human assemblies respectively. Thus, GCE was found to outperform EMR both in terms of cost and wall-clock time. Our findings confirm that cloud computing is an efficient and potentially cost-effective alternative for analysis of large genomic datasets. In addition to releasing our cost-effectiveness comparison, we present available ready-to-use scripts for establishing Hadoop instances with Ganglia monitoring on EC2 or GCE.

Meta-analysis of 2,104 trios provides support for 10 novel candidate genes for intellectual disability

“…the GSMR of the gene TTN). 20 We then calculated specific mutation rates for the two defined functional classes (loss-of-function, 21 functional). The GSMR for loss-of-function DNMs was calculated by summing the individual GSMR for 22 nonsense, splice site and frame-shift variants; The GSMR for functional DNMs was calculated by 23 summing the GSMR for the loss-of-function (LoF) variants with the missense mutation rate; and for 24 genes for which variants from different functional classes were identified, we used the overall GSMR.…”

“…The RVIS ranks genes based on whether they have more or less common 18 functional genetic variation relative to the genome-wide expectation. The initial RVIS gene scores were 19 computed based on the NHLBI-ESP6500 data set 20 and recently recomputed based on the ExAC v0. 3 20 dataset (http://genic-intolerance.org/).…”

Low cost, high performance processing of single particle cryo-electron microscopy data in the cloud

“…Of note, global biotechnology companies such as Novartis, Bristol-Myers-Squibb, and Pfizer have utilized the computing power of Amazon for scientific data processing. Many academic researchers have also begun to use Amazon’s EC2 resources for analyzing such datasets as super-resolution light microscopy images (Hu et al, 2013), genomics (Krampis et al, 2012; Yazar et al, 2014), and proteomics (Mohammed et al, 2012; Trudgian and Mirzaei, 2012).…”

Low cost, high performance processing of single particle cryo-electron microscopy data in the cloud

“…Of note, global biotechnology companies such as Novartis ( AWS, 2014a ), Bristol-Myers-Squibb ( AWS, 2013 ), and Pfizer ( AWS, 2014b ) have utilized the computing power of Amazon for scientific data processing. Many academic researchers have also begun to use Amazon's EC2 resources for analyzing datasets from super-resolution light microscopy ( Hu et al, 2013 ), genomics ( Krampis et al, 2012 ; Yazar et al, 2014 ), and proteomics ( Mohammed et al, 2012 ; Trudgian and Mirzaei, 2012 ).…”