Over the last decade, the introduction of microarray technology has had a profound impact on gene expression research. The publication of studies with dissimilar or altogether contradictory results, obtained using different microarray platforms to analyze identical RNA samples, has raised concerns about the reliability of this technology. The MicroArray Quality Control (MAQC) project was initiated to address these concerns, as well as other performance and data analysis issues. Expression data on four titration pools from two distinct reference RNA samples were generated at multiple test sites using a variety of microarray-based and alternative technology platforms. Here we describe the experimental design and probe mapping efforts behind the MAQC project. We show intraplatform consistency across test sites as well as a high level of interplatform concordance in terms of genes identified as differentially expressed. This study provides a resource that represents an important first step toward establishing a framework for the use of microarrays in clinical and regulatory settings.

Effective use of microarray technology in clinical and regulatory settings is contingent on the adoption of standard methods for assessing performance. The MicroArray Quality Control project evaluated the repeatability and comparability of microarray data on the major commercial platforms and laid the groundwork for the application of microarray technology to regulatory assessments. However, methods for assessing performance that are commonly applied to diagnostic assays used in laboratory medicine remain to be developed for microarray assays. A reference system for microarray performance evaluation and process improvement was developed that includes reference samples, metrics and reference datasets. The reference material is composed of two mixes of four different rat tissue RNAs that allow defined target ratios to be assayed using a set of tissue-selective analytes that are distributed along the dynamic range of measurement. The diagnostic accuracy of detected changes in expression ratios, measured as the area under the curve from receiver operating characteristic plots, provides a single commutable value for comparing assay specificity and sensitivity. The utility of this system for assessing overall performance was evaluated for relevant applications like multi-laboratory proficiency testing programs and single-laboratory process drift monitoring. The diagnostic accuracy of detection of a 1.5-fold change in signal level was found to be a sensitive metric for comparing overall performance. This test approaches the technical limit for reliable discrimination of differences between two samples using this technology. We describe a reference system that provides a mechanism for internal and external assessment of laboratory proficiency with microarray technology and is translatable to performance assessments on other whole-genome expression arrays used for basic and clinical research.

Microarrays can be used to simultaneously measure the differential expression states of many mRNA's in two samples. Such measurements are limited by systematic and random errors. Systematic errors include labeling bias, imperfect feature morphologies, mismatched sample concentrations, and cross-hybridization. Random errors arise from chemical and scanning noise, particularly for low signals. We have used a combination of fluor-exchanged two-color labeling and improved normalization methods to minimize systematic errors from labeling bias, imperfect features, and mismatched sample concentrations. On-array specificity control probes and experimentally proven probe design algorithms were used to correct for cross-hybridization. Random errors were reduced via automated non-uniform feature flagging and an advanced scanner design. We have scored feature significance, using established statistical tests. We have then estimated the intrinsic random measurement error as a function of average probe signal via sample self-comparison experiments (human K-562 cell mRNA). Finally, we have combined all of these tools in the analysis of differential expression measurements between K-562 cells and HeLa cells. The results establish the importance of the elimination of systematic errors and the objective assessment of the effects of random errors in producing reliable estimates of differential expression.

To extract useful biological information from DNA microarray experiments, it is necessary to accurately quantify the measured expression levels and to systemat-