Among the many analytical techniques that have been published to analyze the significance of the effects in the absence of replications, two have emerged as the most widely used in text books as well as statistical software packages: The Lenth's method and the estimation of the variance of the effects from the values of those considered negligible. This article shows that neither is better than the other in all cases, and by analyzing the results obtained in a wide variety of situations it provides guidelines on when it is preferable to use one or the other technique.
The Lenth method is conceptually simple and probably the most common approach to analyzing the significance of the effects in factorial designs. Here, we compare it with a Bayesian approach proposed by Box and Meyer and which does not appear in the usual software packages. The comparison is made by simulating the results of 4, 8 and 16 run designs in a set of scenarios that mirror practical situations and analyzing the results provided by both methods. Although the results depend on the number of runs and the scenario considered, the use of the Box and Meyer method generally produces better results.
Different critical values deduced by simulation have been proposed that greatly improve Lenth's original proposal. However, these simulations assume that all effects are zerosomething not realistic-producing bigger than desired critical values and thus significance levels lower that intended. This article, in accordance with George Box [2] well known idea that Experimental Design should be about learning and not about testing and based on studying how the presence of a realistic number and size of active effects affects critical values, proposes to use t = 2 for any number of runs equal or greater than 8. And it shows that this solution, in addition of being simpler, provides under reasonable realistic situations better results than those obtained by simulation.
This article analyzes the increase in the probability of committing type I and type II errors in assessing the significance of the effects when some properly selected runs have not been carried out, and their responses have been estimated from the interactions considered null from scratch. This is done by simulating the responses from known models that represent a wide variety of practical situations that the experimenter will encounter; the responses considered to be missing are then estimated, and the significance of the effects is assessed. Through comparison with the parameters of the model, the errors are then identified. To assess the significance of the effects when there are missing values, the Box‐Meyer method has been used. The conclusions are that one missing value in eight run designs, and up to three missing values in 16 run designs experiments can be estimated without hardly any notable increase in the probability of error when assessing the significance of the effects.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.