Assessing the Effects of Data Selection and Representation on the Development of Reliable E. coli Sigma 70 Promoter Region Predictors

Abstract

As the number of sequenced bacterial genomes increases, the need for rapid and reliable tools for the annotation of functional elements (e.g., transcriptional regulatory elements) becomes more desirable. Promoters are the key regulatory elements, which recruit the transcriptional machinery through binding to a variety of regulatory proteins (known as sigma factors). The identification of the promoter regions is very challenging because these regions do not adhere to specific sequence patterns or motifs and are difficult to determine experimentally. Machine learning represents a promising and cost-effective approach for computational identification of prokaryotic promoter regions. However, the quality of the predictors depends on several factors including: i) training data; ii) data representation; iii) classification algorithms; iv) evaluation procedures. In this work, we create several variants of E. coli promoter data sets and utilize them to experimentally examine the effect of these factors on the predictive performance of E. coli σ 70 promoter models. Our results suggest that under some combinations of the first three criteria, a prediction model might perform very well on cross-validation experiments while its performance on independent test data is drastically very poor. This emphasizes the importance of evaluating promoter region predictors using independent test data, which corrects for the over-optimistic performance that might be estimated using the cross-validation procedure. Our analysis of the tested models shows that good prediction models often perform well despite how the non-promoter data was obtained. On the other hand, poor prediction models seems to be more sensitive to the choice of non-promoter sequences. Interestingly, the best performing sequence-based classifiers outperform the best performing structure-based classifiers on both cross-validation and independent test performance evaluation experiments. Finally, we propose a meta-predictor method combining two top performing sequence-based and structure-based classifiers and compare its performance with some of the state-of-the-art E. coli σ 70 promoter prediction methods.

Benchmarking Bacterial Promoter Prediction Tools: Potentialities and Limitations

“…As can be seen from the results above, new tools have emerged with enhanced performance compared to widely used ones. Although the best performing tool uses just sequence-based features (a result that corroborates with Abbas et al [ 42 ]), in general, algorithms using feature extraction that combines attributes derived from sequence together with physicochemical properties of DNA achieved better results. It is also clear from our results that choosing the appropriate control (or negative) data set to construct these algorithms is crucial to avoid false-positive results.…”

A novel method for identifying and distinguishing Cryptococcus neoformans and Cryptococcus gattii by surface-enhanced Raman scattering using positively charged silver nanoparticles

“…The average accuracy of the training and test data were obtained after 10 operations. Tables 1 and 2 (Model 2–11) show that the average accuracy of the training and test data are 100% and 100%, respectively, which indicates that the model has good classification ability 26 , 27 .…”

IoMT-Based Association Rule Mining for the Prediction of Human Protein Complexes

“…where True positive is the protein complexes predicted as complexes, False Positive is the non-protein complexes predicted as complexes, False Negative is protein complexes predicted as non-protein complexes [40]. In the statistical prediction, independent data set test, K-fold cross validation test, and jackknife cross-validation are usually used to assess the prediction capability of the model.…”