On the evaluation of the fidelity of supervised classifiers in the prediction of chimeric RNAs

Abstract

BackgroundHigh-throughput sequencing technology and bioinformatics have identified chimeric RNAs (chRNAs), raising the possibility of chRNAs expressing particularly in diseases can be used as potential biomarkers in both diagnosis and prognosis.ResultsThe task of discriminating true chRNAs from the false ones poses an interesting Machine Learning (ML) challenge. First of all, the sequencing data may contain false reads due to technical artifacts and during the analysis process, bioinformatics tools may generate false positives due to methodological biases. Moreover, if we succeed to have a proper set of observations (enough sequencing data) about true chRNAs, chances are that the devised model can not be able to generalize beyond it. Like any other machine learning problem, the first big issue is finding the good data to build models. As far as we were concerned, there is no common benchmark data available for chRNAs detection. The definition of a classification baseline is lacking in the related literature too. In this work we are moving towards benchmark data and an evaluation of the fidelity of supervised classifiers in the prediction of chRNAs.ConclusionsWe proposed a modelization strategy that can be used to increase the tools performances in context of chRNA classification based on a simulated data generator, that permit to continuously integrate new complex chimeric events. The pipeline incorporated a genome mutation process and simulated RNA-seq data. The reads within distinct depth were aligned and analysed by CRAC that integrates genomic location and local coverage, allowing biological predictions at the read scale. Additionally, these reads were functionally annotated and aggregated to form chRNAs events, making it possible to evaluate ML methods (classifiers) performance in both levels of reads and events. Ensemble learning strategies demonstrated to be more robust to this classification problem, providing an average AUC performance of 95 % (ACC=94 %, Kappa=0.87 %). The resulting classification models were also tested on real RNA-seq data from a set of twenty-seven patients with acute myeloid leukemia (AML).Electronic supplementary materialThe online version of this article (doi:10.1186/s13040-016-0112-6) contains supplementary material, which is available to authorized users.

SimBA: A methodology and tools for evaluating the performance of RNA-Seq bioinformatic pipelines

“…This is significant as incorrect or ambiguous alignment can, by itself, generate substantial numbers of false fusions. We selected two mappers optimized for the detection of fusions; CRAC with the integration of a model based on machine learning [ 25 ] and STAR that recently added a procedure to identify reads associated with fusions. We varied two parameters in these tools, CRAC’s chim_value and STAR’s chim_segment .…”

“…We then created multiple versions of CRAC_fusion by filtering fusions based on their chim_values from 0 to 1 (0, 0.2, 0.4, 0.6, 0.8, 0.85, 0.9, 0.95, 1). This options controls the algorithmic quality of chimeric junctions detected by CRAC based on a machine learning procedure [ 25 ].…”

Transcriptional-Readthrough RNAs Reflect the Phenomenon of “A Gene Contains Gene(s)” or “Gene(s) within a Gene” in the Human Genome, and Thus Are Not Chimeric RNAs

“…On the other hand, more publications continue emerging to report [ 95 ] or summarize [ 3 , 69 , 70 ] such trans -splicing related chimeras or other noncolinear RNAs. Moreover, many bioinformatic experts are establishing different algorithms to cull chimeras from different sets of high-throughput sequencing data [ 96 , 97 , 98 , 99 , 100 , 101 , 102 , 103 , 104 ], although all these data sets contain many spurious sequences, as we and others have pointed out [ 5 , 6 , 64 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 ]. This situation is worrisome to us.…”

“…For example, splicing is initiated and finished too quickly to study its detail, as aforementioned. In addition, the reported detection of the abovementioned RNAs all involved reverse transcription (RT) and polymerase chain reactions (PCR), which are techniques that easily create spurious results, as we and others have repeatedly described before, due to template-switching, mis-priming, self-priming, DNA or complementary DNA (cDNA) damage, and PCR-reconditioning, among other reasons [ 5 , 6 , 64 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 ]. Therefore, approaches without involvement of RT and PCR are needed to minimize technical artifacts for indisputable evidence and to obtain procedural and mechanistic details of the presumed trans -splicing.…”

“…We should not assume that all two-neighboring-gene RNAs are produced by transcriptional readthrough simply because readthrough is common, while arbitrarily ruling out the possible involvement of trans -splicing that is also considered by other researchers to be a common event [ 15 , 66 , 67 , 68 , 69 , 70 , 71 ]. A caveat probably needs to be given that convincing experimental proof should require a non-RT and non-PCR approach to avoid technical spuriousness that may be created by these techniques [ 5 , 6 , 64 , 105 , 106 , 107 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 , 123 , 124 , 125 , 126 , 127 , 128 , 129 , 130 , 131 , 132 , 133 , 134 , 135 ], by using the cDNA protection assay established by us [ 105 ], the less sensitive RNA protection assay [ 136 , 137 , 138 ], or other approaches [ 110 , 111 ] as alternatives.…”