Simultaneous estimation of normal means with side information

Abstract

The integrative analysis of multiple datasets is an important strategy in data analysis. It is increasingly popular in genomics, which enjoys a wealth of publicly available datasets that can be compared, contrasted, and combined in order to extract novel scientific insights. This paper studies a stylized example of data integration for a classical statistical problem: leveraging side information to estimate a vector of normal means. This task is formulated as a compound decision problem, an oracle integrative decision rule is derived, and a data-driven estimate of this rule based on minimizing an unbiased estimate of its risk is proposed. The data-driven rule is shown to asymptotically achieve the minimum possible risk among all separable decision rules, and it can outperform existing methods in numerical properties. The proposed procedure leads naturally to an integrative high-dimensional classification procedure, which is illustrated by combining data from two independent gene expression profiling studies.

A nonparametric regression approach to asymptotically optimal estimation of normal means

“…For example, because both the optimal separable estimator (1), d * (x), and the nonparametric g-modeling estimators can be thought of as Gaussian posterior means, they have infinite, continuous, bounded derivatives; however, this property is not satisfied by all functions in the two function classes we studied. It is likely that estimating d * (x) using a smaller function class of smoother functions would improve the performance of our methodology; this is supported by the performance of the use of empirical risk minimization to estimate d * (x) directly [60].…”

“…The use of regression regularization strategies, such as shape constraints, allows us to derive asymptotically optimal estimators without appealing to Bayesian arguments. We hope that our results will serve as a foundation for the development of the regression approach in settings where empirical Bayes methods struggle, such as the designing of non-separable estimators [21,61] and the incorporation of side information [4,60]. We also hope that our approach can lead to novel estimators with distinct advantages for frequentist questions such as confidence intervals and inference.…”

“…Related approaches have been considered for this problem before. Many empirical Bayesian methods such as [59,58,60,3] use empirical risk estimation to fit their estimators. These methods get their parameterization from Tweedie's formula (2) and often utilize a secondary metric, such as marginal likelihood, as their fitting criteria.…”

“…Johnstone and Silverman [29] introduced this simulation setting to compare their posterior median method to many false discovery rate methods and various thresholding methods. Many other papers including [27,11,32,60,31] have extended this Johnstone and Silverman's Table 1 to study their estimators, making this simulation setting a standard performance metric for this simultaneous estimation problem. In Table 1 we continue the pattern of using the sparse model setting to study our estimators.…”