Evaluating the accuracy of SHAPE-directed RNA secondary structure predictions

Abstract

Recent advances in RNA structure determination include using data from high-throughput probing experiments to improve thermodynamic prediction accuracy. We evaluate the extent and nature of improvements in data-directed predictions for a diverse set of 16S/18S ribosomal sequences using a stochastic model of experimental SHAPE data. The average accuracy for 1000 data-directed predictions always improves over the original minimum free energy (MFE) structure. However, the amount of improvement varies with the sequence, exhibiting a correlation with MFE accuracy. Further analysis of this correlation shows that accurate MFE base pairs are typically preserved in a data-directed prediction, whereas inaccurate ones are not. Thus, the positive predictive value of common base pairs is consistently higher than the directed prediction accuracy. Finally, we confirm sequence dependencies in the directability of thermodynamic predictions and investigate the potential for greater accuracy improvements in the worst performing test sequence.

Data-directed RNA secondary structure prediction using probabilistic modeling

“…There are two rationales behind this transformation. First, it induces Gaussianity for each structural context we considered, obviating the need for fitting distinct density functions on a per-context basis (Sükösd et al 2013). Second, it increases homoskedasticity as a consequence of the log-log relationship between reactivities and associated variances as shown by (K Choudhary, NP Shih, F Deng, M Ledda, B Li, S Aviran, in prep.).…”

“…To assess confidence in our performance evaluations in the absence of replicate data, we resorted to simulating replicates for each RNA using a previous model and methodology (Sükösd et al 2013). Briefly speaking, we randomly sampled reactivities from modeled probability densities associated with structural contexts.…”

“…Normal distributions N (μ, σ) were fitted for both logarithm (Ln) and Box-Cox transformed data. The parameters for the exponential decay and GEV distributions were derived from Sükösd et al (2013).…”

“…However, with increasing availability of data, it becomes feasible to incorporate more refined contexts with different statistical properties, which may improve prediction accuracy. For example, bases at helix ends tend to be more reactive than those in stacked pairs (Sükösd et al 2013).…”

“…Alternatively, the data were fit to a known parametric density, from which P(a i |p i ) was calculated. In this study, we followed previous work (Sükösd et al 2013), where unpaired data were fit to an exponential distribution, and data of helix-end and stacked bases were each fit to a generalized extreme value (GEV) distribution (Fig. 2, upper panel).…”