Experimental and Statistical Considerations to Avoid False Conclusions in Proteomics Studies Using Differential In-gel Electrophoresis

Abstract

In quantitative proteomics, the false discovery rate (FDR) can be defined as the number of false positives within statistically significant changes in expression. False positives accumulate during the simultaneous testing of expression changes across hundreds or thousands of protein or peptide species when univariate tests such as the Student's t test are used. Currently most researchers rely solely on the estimation of p values and a significance threshold, but this approach may result in false positives because it does not account for the multiple testing effect. For each species, a measure of significance in terms of the FDR can be calculated, producing individual q values. The q value maintains power by allowing the investigator to achieve an acceptable level of true or false positives within the calls of significance. The q value approach relies on the use of the correct statistical test for the experimental design. In this situation, a uniform p value frequency distribution when there are no differences in expression between two samples should be obtained. Here we report a bias in p value distribution in the case of a three-dye DIGE experiment where no changes in expression are occurring. The bias was shown to arise from correlation in the data from the use of a common internal standard. With a two-dye schema, where each sample has its own internal standard, such bias was removed, enabling the application of the q value to two different proteomics studies. In the case of the first study, we demonstrate that 80% of calls of significance by the more traditional method are false positives. In the second, we show that calculating the q value gives the user control over the FDR. These studies demonstrate the power and ease of use of the q value in correcting for multiple testing. This work also highlights the need for robust experimental design that includes the appropriate application of statistical procedures. Molecular & Cellular Proteomics 6: 1354 -1364, 2007.Quantitative proteomics, the study of global changes in protein expression, is a rapidly growing field where two-dimensional (2D) 1 gel electrophoresis (1, 2), differential labeling of protein and peptides with stable isotopes (3), and label-free mass spectrometric peak intensity measurements (4) are pivotal approaches to measuring changes in the expression level of proteins. The output of large scale genomic sequencing and gene expression studies have driven the need to globally assess protein behavior, which is central to our understanding of cellular function and disease processes. In quantitative studies, the quantitation of the protein or peptide signal allows the comparison of protein levels from one state with another. Regardless of the technique used, the data generated can be analyzed with a variety of statistical approaches. In a simple comparison of two samples, changes in protein expression are considered to be significant when above a specified threshold (5). More rigorous studies incorporating replicates use univariate (1, 6) or multivari...

DIGE analysis of rat skeletal muscle proteins using nonionic detergent phase extraction of young adult versus aged gastrocnemius tissue

“…For the comparative proteomic analysis of proteins fractionated into AQ phase and DT phase, minimal fluorescent dye labeling was performed at a ratio of 25 µg of protein per 200 pmol of CyDye as previously described [30], following the recommendations of Karp and co-workers for optimized DIGE analysis [31,32]. Labeled samples were incubated on ice for 30 min in the dark.…”

Quantitative Proteomic Analysis Reveals Metabolic Alterations, Calcium Dysregulation, and Increased Expression of Extracellular Matrix Proteins in Laminin α2 Chain–deficient Muscle

“…Still, compared proteins should present a variation coefficient of less than 25% and a protein ratio less than 1.25 or greater than 1.25. In order to avoid multiple hypothesis test error, q-values were estimated (27,28). Also, to ensure the reproducibility of our findings, the retrospective statistical power was calculated using the free statistical software package R, according to Ref.…”

Statistical Model to Analyze Quantitative Proteomics Data Obtained by 18O/16O Labeling and Linear Ion Trap Mass Spectrometry

“…The majority of published quantitative proteomics studies have used the established two-dimensional gel electrophoresis approach and use the p value to identify protein species showing significant expression changes. The need to use robust experimental designs (19) and to control for FDR in differential electrophoresis studies has been claimed only recently (20). The majority of published DIGE studies use the Student's t test to identify significant changes in expression (20); however, this test assumes independent sampling, normality of the data, and homogeneity of the variance, and the validity of these assumptions has not been demonstrated in a general context.…”

“…The need to use robust experimental designs (19) and to control for FDR in differential electrophoresis studies has been claimed only recently (20). The majority of published DIGE studies use the Student's t test to identify significant changes in expression (20); however, this test assumes independent sampling, normality of the data, and homogeneity of the variance, and the validity of these assumptions has not been demonstrated in a general context. The situation is not better with expression studies obtained using stable isotope labeling approaches.…”