Integrated Systems for NGS Data Management and Analysis: Open Issues and Available Solutions

Abstract

Next-generation sequencing (NGS) technologies have deeply changed our understanding of cellular processes by delivering an astonishing amount of data at affordable prices; nowadays, many biology laboratories have already accumulated a large number of sequenced samples. However, managing and analyzing these data poses new challenges, which may easily be underestimated by research groups devoid of IT and quantitative skills. In this perspective, we identify five issues that should be carefully addressed by research groups approaching NGS technologies. In particular, the five key issues to be considered concern: (1) adopting a laboratory management system (LIMS) and safeguard the resulting raw data structure in downstream analyses; (2) monitoring the flow of the data and standardizing input and output directories and file names, even when multiple analysis protocols are used on the same data; (3) ensuring complete traceability of the analysis performed; (4) enabling non-experienced users to run analyses through a graphical user interface (GUI) acting as a front-end for the pipelines; (5) relying on standard metadata to annotate the datasets, and when possible using controlled vocabularies, ideally derived from biomedical ontologies. Finally, we discuss the currently available tools in the light of these issues, and we introduce HTS-flow, a new workflow management system conceived to address the concerns we raised. HTS-flow is able to retrieve information from a LIMS database, manages data analyses through a simple GUI, outputs data in standard locations and allows the complete traceability of datasets, accompanying metadata and analysis scripts.

An early Myc‐dependent transcriptional program orchestrates cell growth during B‐cell activation

“…Their quality was evaluated and confirmed using the FastQC application: (http://www.bioinformatics.babraham.ac.uk/projects/fastqc/). Pipelines for primary analysis (filtering and alignment to the reference genome of the raw reads) and secondary analysis (expression quantification, differential gene expression, and peak calling) have been integrated in the HTS‐flow system . Bioinformatic and statistical analysis were performed using R with Bioconductor and comEpiTools packages .…”

“…bioinformatics.babraham.ac.uk/projects/fastqc/). Pipelines for primary analysis (filtering and alignment to the reference genome of the raw reads) and secondary analysis (expression quantification, differential gene expression, and peak calling) have been integrated in the HTS-flow system [52]. Bioinformatic and statistical analysis ª 2019 The Authors EMBO reports 20: e47987 | 2019…”

No wisdom in the crowd: genome annotation in the era of big data – current status and future prospects

“…Genome sequence data are strings of the four DNA nucleotide bases. Experimentally obtained, they are associated with a first range of metadata such as sequence quality, fragment length or methodology- oriented paired-ends sequencing (Bianchi et al, 2016). Subsequently, when sequences begin to be organized with the aim of specific biological understanding, we have long reads, contigs and also repeats, G+C content, tetranucleotide frequencies and a plethora of sequence descriptors [see (Weinel et al, 2002) as an example].…”

Big Data Science and its Applications in Healthcare and Medical Research: Challenges and Opportunities

“…For instance, in the genomic domain, for Affymetrix time course data obtained from Affymetrix GeneChips, one may use Affymetrix software (MAS 5.0) and probe set algorithms of MAS5 for background subtraction, signal intensity normalization between arrays, and non-specific hybridization correction etc [75][76][77][78][79]. To do so, high level performance hardware and software (e.g., programming languages and algorithms for visualizations) that conduct parallel and distributed and cloud computing to manage, retrieve, reformat and analyze the data from various resources including the genomic laboratory and hospital patient information systems needs be considered (Table 1) [58,59,[80][81][82][83][84]. For instance, Bianchi et al [81] developed HTS-flow, a workflow management system that can retrieve information from a laboratory management system database, manages Omics data analyses through a simple GUI, outputs data in standard locations and allows the complete traceability of datasets, accompanying metadata and analysis scripts.…”

“…To do so, high level performance hardware and software (e.g., programming languages and algorithms for visualizations) that conduct parallel and distributed and cloud computing to manage, retrieve, reformat and analyze the data from various resources including the genomic laboratory and hospital patient information systems needs be considered (Table 1) [58,59,[80][81][82][83][84]. For instance, Bianchi et al [81] developed HTS-flow, a workflow management system that can retrieve information from a laboratory management system database, manages Omics data analyses through a simple GUI, outputs data in standard locations and allows the complete traceability of datasets, accompanying metadata and analysis scripts. Childs et al [82] designed and implemented SoFIA, an Omics data integration framework for annotating high throughput data sets [82].…”