A parametric bootstrap approach for ANOVA with unequal variances: Fixed and random models

Krishnamoorthy, K.; Lu, Fei; Mathew, Thomas

doi:10.1016/j.csda.2006.09.039

Cited by 139 publications

(113 citation statements)

References 17 publications

Supporting

Mentioning

106

Contrasting

Unclassified

Order By: Relevance

“…That is the most important factor goes first since our factors are not necessarily orthogonal so some variance might conceivably be allocated to one of several factors depending upon the ordering of the factors in the model. As is conventional ( [40], [38], [41], [42]) we construct the model in decreasing order of partial eta squared scores (see Table 13). All four factors are significant.…”

Section: Anova Model Resultsmentioning

confidence: 99%

Researcher Bias: The Use of Machine Learning in Software Defect Prediction

Shepperd

Bowes

Hall

2014

IIEEE Trans. Software Eng.

288

190

View full text Add to dashboard Cite

Abstract-Background. The ability to predict defect-prone software components would be valuable. Consequently, there have been many empirical studies to evaluate the performance of different techniques endeavouring to accomplish this effectively. However no one technique dominates and so designing a reliable defect prediction model remains problematic. Objective. We seek to make sense of the many conflicting experimental results and understand which factors have the largest effect on predictive performance. Method. We conduct a meta-analysis of all relevant, high quality primary studies of defect prediction to determine what factors influence predictive performance. This is based on 42 primary studies that satisfy our inclusion criteria that collectively report 600 sets of empirical prediction results. By reverse engineering a common response variable we build a random effects ANOVA model to examine the relative contribution of four model building factors (classifier, data set, input metrics and researcher group) to model prediction performance. Results. Surprisingly we find that the choice of classifier has little impact upon performance (1.3%) and in contrast the major (31%) explanatory factor is the researcher group. It matters more who does the work than what is done. Conclusion. To overcome this high level of researcher bias, defect prediction researchers should (i) conduct blind analysis, (ii) improve reporting protocols and (iii) conduct more intergroup studies in order to alleviate expertise issues. Lastly, research is required to determine whether this bias is prevalent in other applications domains.

show abstract

Section: Anova Model Resultsmentioning

confidence: 99%

Researcher Bias: The Use of Machine Learning in Software Defect Prediction

Shepperd

Bowes

Hall

2014

IIEEE Trans. Software Eng.

288

190

View full text Add to dashboard Cite

show abstract

“…Bu durumda klasik F testinin kullanılması uygun değildir. Bu amaçla literatürde yığın varyansları homojen olmadığı zaman normal dağılıma sahip k sayıda yığınların ortalamasının eşitliği için birçok test geliştirilmiştir [2][3][4][5][6][7].…”

Section: Introductionunclassified

“…Genelleştirilmiş p değeri, parametrik bootstrap ve hesaplamalı yaklaşım testi gibi yeniden örneklemeye dayalı yöntemler özellikle ilgilenilen parametrelerin yanı sıra ilgilenilmeyen parametreleri de içeren problemleri çözmek için oldukça sık kullanılmaktadır [4,5,[23][24][25][26][27][28]. …”

Section: Introductionunclassified

Varyansların Homojenliği için Kullanılan Yeniden Örneklemeye Dayalı Testler ve Simülasyon Çalışması

Gökpınar¹

2017

SDÜ Fen Bil Enst Der

View full text Add to dashboard Cite

Anahtar KelimelerVaryans homojenliği, Yeniden örnekleme, Simülasyon çalışması Özet: Bu çalışmada normal dağılıma sahip k sayıda yığının varyans homojenliği testi üzerinde durulmuş ve son yıllarda yaygın olarak kullanılan yeniden örnekleme yöntemlerine dayalı testler incelenmiştir. Simülasyon çalışmasında testlerin performansını belirlemek amacıyla tüm testler deneysel I. tip hata oranı ve güç bakımından karşılaştırılmıştır. Bu amaçla farklı grup sayısı, farklı örnek çapı ve farklı yığın varyansları alınarak bir simülasyon çalışması yapılmış ve sonuçları yorumlanmıştır. Resampling Based Tests for the Equality of Variances and Simulation Study Keywords The equality of variances, Resampling methods, Monte CarloAbstract: In this study, we were interested in testing the equality of variances of k normal populations and examined commonly used resampling based tests. In simulation study, all the tests were compared in terms of type I error rate and power to assess the performance of tests. For this purpose, by using different combination of sample sizes, number of groups and population variances comprehensive simulation study was presented and simulation results were interpreted.

show abstract

“…For a balanced case and k = 3 the sample sizes were given as (5, 5, 5), (10,10,10); and for unbalanced (2,3,3), and (5, 7, 15). For k = 5, the sample sizes were (2, 2, 2, 2, 2), (5,5,5,5,5), (10,10,10,10,10), (2,2,3,3,5), (4,4,6,6,10), and (3,5,7,10,10). For parametric bootstrap, for each of the 10000 simulations, 5000 samples were generated and the proportion of those exceeding the observed f * was recorded as the p-value.…”

Section: Simulation Studymentioning

confidence: 99%

“…The parametric bootstrap method, as described in detail in Krishnamoorthy et al [7], involves resampling from a distribution whose parameters are the sample means and sample variances. The reference value for the test is the observed value f * of F * .…”

Section: Parametric Bootstrapmentioning

confidence: 99%

Accuracy of p-values of approximate tests in testing for equality of means under unequal variances

Vólaufová

2009

Mathematica Slovaca

View full text Add to dashboard Cite

ABSTRACT. Seemingly, testing for fixed effects in linear models with variancecovariance components has been solved for decades. However, even in simple situations such as in fixed one-way model with heteroscedastic variances (a multiple means case of the Behrens-Fisher problem) the questions of statistical properties of various approximations of test statistics are still alive. Here we present a brief overview of several approaches suggested in the literature as well as those available in statistical software, accompanied by a simulation study in which the accuracy of p-values is studied. Our interest is limited here to the Welch's test, the Satterthwaite-Fai-Cornelius test, the Kenward-Roger test, the simple ANOVA F -test, and the parametric bootstrap test. We conclude that for small sample sizes, regardless the number of compared means and the heterogeneity of variance, the ANOVA F -test p-value performs the best. For higher sample sizes (at least 5 per group), the parametric bootstrap performs well, and the Kenward-Roger test also performs well.

show abstract

A parametric bootstrap approach for ANOVA with unequal variances: Fixed and random models

Cited by 139 publications

References 17 publications

Researcher Bias: The Use of Machine Learning in Software Defect Prediction

Researcher Bias: The Use of Machine Learning in Software Defect Prediction

Varyansların Homojenliği için Kullanılan Yeniden Örneklemeye Dayalı Testler ve Simülasyon Çalışması

Accuracy of p-values of approximate tests in testing for equality of means under unequal variances

Contact Info

Product

Resources

About