Assessing the validity and reproducibility of genome-scale predictions

Sugden, Lauren Alpert; Tackett, Michael; Savva, Yiannis A.; Thompson, W. A.; Lawrence, Charles E.

doi:10.1093/bioinformatics/btt508

Cited by 8 publications

(2 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The assertion that many published scientific studies cannot be reproduced after several studies attempted to reproduce them (Ioannidis et al, 2009;Prinz, Schlange & Asadullah, 2011;Nekrutenko & Taylor, 2012;Begley & Ellis, 2012;Pimentel et al, 2019;Raff, 2019), has recently led the scientific community to look into the problem more seriously. Several reports have raised reproducibility concerns in genetics (Hunt et al, 2012;Surolia et al, 2010), genomics (DeVeale, Van Der Kooy & Babak, 2012Sugden et al, 2013), and oncology (Begley & Ellis, 2012). While the reproduction efforts have often focused on biology, medicine, and psychology, the recent survey by Nature (Baker, 2016a) has shown the problem is widespread and not just pertains to specific fields (Henderson, 2017).…”

Section: Related Workmentioning

confidence: 99%

Understanding experiments and research practices for reproducibility: an exploratory study

Samuel¹,

König‐Ries²

2021

PeerJ

View full text Add to dashboard Cite

Scientific experiments and research practices vary across disciplines. The research practices followed by scientists in each domain play an essential role in the understandability and reproducibility of results. The “Reproducibility Crisis”, where researchers find difficulty in reproducing published results, is currently faced by several disciplines. To understand the underlying problem in the context of the reproducibility crisis, it is important to first know the different research practices followed in their domain and the factors that hinder reproducibility. We performed an exploratory study by conducting a survey addressed to researchers representing a range of disciplines to understand scientific experiments and research practices for reproducibility. The survey findings identify a reproducibility crisis and a strong need for sharing data, code, methods, steps, and negative and positive results. Insufficient metadata, lack of publicly available data, and incomplete information in study methods are considered to be the main reasons for poor reproducibility. The survey results also address a wide number of research questions on the reproducibility of scientific results. Based on the results of our explorative study and supported by the existing published literature, we offer general recommendations that could help the scientific community to understand, reproduce, and reuse experimental data and results in the research data lifecycle.

show abstract

Section: Related Workmentioning

confidence: 99%

Understanding experiments and research practices for reproducibility: an exploratory study

Samuel¹,

König‐Ries²

2021

PeerJ

View full text Add to dashboard Cite

show abstract

“…To identify and adjust for unanticipated sample-wise correlations, Efron (2009) proposed an empirical Bayes approach utilizing the sample moments of the data. In particular, sample-wise correlations may lead to inflated evidence for mean differences, and could be one explanation for the claimed lack of reproducibility in genomics research (Leek et al, 2010;Allen and Tibshirani, 2012;Sugden et al, 2013).…”

Section: Introductionmentioning

confidence: 99%

Joint Mean and Covariance Estimation with Unreplicated Matrix-Variate Data

Hornstein

Fan

Shedden

et al. 2018

Journal of the American Statistical Association

View full text Add to dashboard Cite

It has been proposed that complex populations, such as those that arise in genomics studies, may exhibit dependencies among observations as well as among variables. This gives rise to the challenging problem of analyzing unreplicated high-dimensional data with unknown mean and dependence structures. Matrixvariate approaches that impose various forms of (inverse) covariance sparsity allow flexible dependence structures to be estimated, but cannot directly be applied when the mean and covariance matrices are estimated jointly. We present a practical method utilizing generalized least squares and penalized (inverse) covariance estimation to address this challenge. We establish consistency and obtain rates of convergence for estimating the mean parameters and covariance matrices. The advantages of our approaches are: (i) dependence graphs and covariance structures can be estimated in the presence of unknown mean structure, (ii) the mean structure becomes more efficiently estimated when accounting for the dependence structure among observations; and (iii) inferences about the mean parameters become correctly calibrated. We use simulation studies and analysis of genomic data from a twin study of ulcerative colitis to illustrate the statistical convergence and the performance of our methods in practical settings. Several lines of evidence show that the test statistics for differential gene expression produced by our methods are correctly calibrated and improve power over conventional methods.By the above decompositions, it follows that SpB, J 0 , J 1 q can be expressed as SpB, J 0 , J 1 q " S II`SIII`S T

show abstract

Genome-Wide Analysis of A-to-I RNA Editing

Savva

Laurent

Reenan

2016

Methods in Molecular Biology

View full text Add to dashboard Cite

Adenosine (A)-to-inosine (I) RNA editing is a fundamental posttranscriptional modification that ensures the deamination of A-to-I in double-stranded (ds) RNA molecules. Intriguingly, the A-to-I RNA editing system is particularly active in the nervous system of higher eukaryotes, altering a plethora of noncoding and coding sequences. Abnormal RNA editing is highly associated with many neurological phenotypes and neurodevelopmental disorders. However, the molecular mechanisms underlying RNA editing-mediated pathogenesis still remain enigmatic and have attracted increasing attention from researchers. Over the last decade, methods available to perform genome-wide transcriptome analysis, have evolved rapidly. Within the RNA editing field researchers have adopted next-generation sequencing technologies to identify RNA-editing sites within genomes and to elucidate the underlying process. However, technical challenges associated with editing site discovery have hindered efforts to uncover comprehensive editing site datasets, resulting in the general perception that the collections of annotated editing sites represent only a small minority of the total number of sites in a given organism, tissue, or cell type of interest. Additionally to doubts about sensitivity, existing RNA-editing site lists often contain high percentages of false positives, leading to uncertainty about their validity and usefulness in downstream studies. An accurate investigation of A-to-I editing requires properly validated datasets of editing sites with demonstrated and transparent levels of sensitivity and specificity. Here, we describe a high signal-to-noise method for RNA-editing site detection using single-molecule sequencing (SMS). With this method, authentic RNA-editing sites may be differentiated from artifacts. Machine learning approaches provide a procedure to improve upon and experimentally validate sequencing outcomes through use of computationally predicted, iterative feedback loops. Subsequent use of extensive Sanger sequencing validations can generate accurate editing site lists. This approach has broad application and accurate genome-wide editing analysis of various tissues from clinical specimens or various experimental organisms is now a possibility.

show abstract

Assessing the validity and reproducibility of genome-scale predictions

Abstract: Downloadable software and a web service for both the analysis of data from a reproducibility study and for the optimal design of these studies is provided at http://ccmbweb.ccv.brown.edu/reproducibility.html .

Cited by 8 publications

References 62 publications

Understanding experiments and research practices for reproducibility: an exploratory study

Understanding experiments and research practices for reproducibility: an exploratory study

Joint Mean and Covariance Estimation with Unreplicated Matrix-Variate Data

Genome-Wide Analysis of A-to-I RNA Editing

Contact Info

Product

Resources

About