Minimax Efficient Random Experimental Design Strategies With Application to Model-Robust Design for Prediction

Abstract

In game theory and statistical decision theory, a random (i.e., mixed) decision strategy often outperforms a deterministic strategy in minimax expected loss. As experimental design can be viewed as a game pitting the Statistician against Nature, the use of a random strategy to choose a design will often be beneficial. However, the topic of minimax-efficient random strategies for design selection is mostly unexplored, with consideration limited to Fisherian randomization of the allocation of a predetermined set of treatments to experimental units. Here, for the first time, novel and more flexible random design strategies are shown to have better properties than their deterministic counterparts in linear model estimation and prediction, including stronger bounds on both the expectation and survivor function of the loss distribution. Design strategies are considered for three important statistical problems: (i) parameter estimation in linear potential outcomes models, (ii) point prediction from a correct linear model, and (iii) global prediction from a linear model taking into account an L 2 -class of possible model discrepancy functions. The new random design strategies proposed for (iii) give a finite bound on the expected loss, a dramatic improvement compared to existing deterministic exact designs for which the expected loss is unbounded. Supplementary materials for this article are available online.

Efficiency Requires Adaptation

“…If n mod 3 = 1 then ξ G places (n − 1)/3 observations on each support point and arbitrarily assigns the remaining observation to one of -1, 0 or 1. Waite and Woods (2021) show that a strategy where (n − 1)/3 observations are placed on each support point and the remaining observation is randomly assigned, with probability 1/3, to -1, 0 or 1 outperforms the deterministic approach. Let π G represent the minimax randomized design.…”

“…factorial, optimal and minimax designs] and designs selected via data independent random processes [e.g. Wu (1981); Li (1983); Waite and Woods (2021)].…”

A convex approach to optimum design of experiments with correlated observations

“…We will thus for comparison purposes use the same method for our approach, but note that there is no simple way to extend this to higher dimensions. Instead, random sampling according to the measure, according to the random design strategies defined in Waite and Woods (2020), multiple (say 100) times and selecting the sampled design with the best criterion value has in our experience usually led to even more efficient designs.…”

Optimal minimax random designs for weighted least squares estimators

“…This seminal paper started a line of work where model misspecification is accounted for in order to compute optimal designs. A review of this body of literature can be found in Wiens (2015); see also the recent work of Waite and Woods (2021). A major theme in this literature is that the bias term is more important than the variance, and therefore one can focus on designs that minimize the bias.…”

“…Another important consideration is the randomness of the X's. While it was already mentioned in Wiens (1992) that deterministic designs have infinite minimax risk, random experimental designs were investigated only recently (Waite and Woods, 2021). The latter work shows that random design strategies have smaller minimax error than their deterministic counterparts under linear models.…”