Use of application containers and workflows for genomic data analysis

Abstract

Background:The rapid acquisition of biological data and development of computationally intensive analyses has led to a need for novel approaches to software deployment. In particular, the complexity of common analytic tools for genomics makes them difficult to deploy and decreases the reproducibility of computational experiments.Methods:Recent technologies that allow for application virtualization, such as Docker, allow developers and bioinformaticians to isolate these applications and deploy secure, scalable platforms that have the potential to dramatically increase the efficiency of big data processing.Results:While limitations exist, this study demonstrates a successful implementation of a pipeline with several discrete software applications for the analysis of next-generation sequencing (NGS) data.Conclusions:With this approach, we significantly reduced the amount of time needed to perform clonal analysis from NGS data in acute myeloid leukemia.

A Review of Scalable Bioinformatics Pipelines

“…With analyses provided increasingly as a service, we believe multi-user job optimizations will become increasingly important. Staggered execution of multiple pipeline jobs can also improve resource utilization as shown in [17,27].…”

“…Both the Dockerfile and the resulting container can be moved between machines without installing additional software, and the container can be rerun later with the exact same libraries. Containers can be orchestrated for parallel execution using, for example, Kubernetes (https://kubernetes.io/) or Docker Swarm (https://github.com/docker/swarm), and there are now multiple pipelining tools that use Docker or provide Docker container support including Nextflow [26], Toil [22], Pachyderm (http://www.pachyderm.io/), Luigi (https:// github.com/spotify/luigi) [27], Rabix/bunny [28], and our own walrus system (http://github.com/fjukstad/walrus).…”

Reproducible Bioconductor Workflows Using Browser-based Interactive Notebooks and Containers

“…Docker is a well-established container technology to increase reproducibility of bioinformatics workflows [3 10 13]. The Docker platform allows for virtualized application deployments within a lightweight, Linux-based wrapper or container [14]. Essentially, containers are virtual environments that encapsulate only the minimum necessary dependencies, which can be quickly deployed on most major platforms in a reproducible fashion [14].…”

Approaches for containerized scientific workflows in cloud environments with applications in life science

“…There are also initiatives for standardizing containers in the life sciences such as BioContainers [11] . Containers have seen an increasing uptake in the life sciences, both for delivering software tools and for facilitating data analysis in various ways [12][13][14][15][16] .…”