A comparison of design and model selection methods for supersaturated experiments

Abstract

Various design and model selection methods are available for supersaturated designs having more factors than runs but little research is available on their comparison and evaluation. In this paper, simulated experiments are used to evaluate the use of E(s2)-optimal and Bayesian D-optimal designs, and to compare three analysis strategies representing regression, shrinkage Various design and model selection methods are available for supersaturated designs having more factors than runs but little research is available on their comparison and evaluation. In this paper, simulated experiments are used to evaluate the use of E(s 2 )-optimal and Bayesian D-optimal designs, and to compare three analysis strategies representing regression, shrinkage and a novel model-averaging procedure. Suggestions are made for choosing the values of the tuning constants for each approach. Findings include that (i) the preferred analysis is via shrinkage; (ii) designs with similar numbers of runs and factors can be effective for a considerable number of active effects of only moderate size; and (iii) unbalanced designs can perform well. Some comments are made on the performance of the design and analysis methods when effect sparsity does not hold.

Design of Experiments for Screening

“…Both E(s 2 )-and D-optimal designs can be found numerically, using algorithms such as columnwisepairwise [59] or coordinate exchange [71]. From simulation studies, it has been shown that there is little difference in the performance of E(s 2 )-and Bayesian D-optimal designs assessed by, for example, sensitivity and type I error rate [67].…”

“…, β d , with the value of τ 2 reflecting the quantity of available prior information. However, the optimal designs obtained tend to be insensitive to the choice of τ 2 [67]. Both E(s 2 )-and D-optimal designs can be found numerically, using algorithms such as columnwisepairwise [59] or coordinate exchange [71].…”

“…The designs in Tables 4 and 5 [67,52] by including the inner-product between each column of X n and the vector 1 n , the n × 1 vector with every entry 1, which corresponds to the intercept term in model (1). This extension widens the class of available designs.…”

“…Consequently, there has been some controversy about the use of these designs [1]. Recently, evidence has been provided for the effectiveness of the designs when factor sparsity holds and the active main effects are large [67,34]. Applications of supersaturated designs include screening variables in numerical models for circuit design [64], extraterrestrial atmospheric science [27] and simulation models for maritime terrorism [114].…”

Selection of active effects based on a ridge regression in supersaturated response surface designs

“…Recently, Li et al (2010) proposed variable selection methods based on cluster analysis, Edwards and Mee (2011) constructed a randomization test method, and Koukouvinos et al (2011) proposed using entropy measures for variable selection. Other methods for the data analysis of supersaturated designs have been discussed by, for example, Abraham et al (1999), Beattie et al (2002), Li and Lin (2003), Cossari (2008), and Marley and Woods (2010). See Georgiou (2014), in which a review of the construction and analysis of supersaturated designs was provided.…”