The Importance of Type III and Type IV Epistemic Errors for Improving Empirical Science

Abstract

Sciences that use the null hypothesis statistical test continue to contend with type I errors (false positives) and type II errors (false negatives). In addition to those errors, statisticians and researchers have identified type III and type IV errors, which focus a scientist’s attention on the larger logical and epistemic outcomes of statistical decision-making that accompany either rejecting or retaining the null hypothesis. Specifically, type III and IV errors interrogate the match between theory and measurement on one side and statistical procedures on the other side. Type III errors occur when scientists use the correct statistical procedures on the wrong theoretical organization or operationalization of variables (viz., theory misspecification). In contrast, type IV errors occur when scientists use the incorrect statistical procedures (viz., evaluation misspecification) on the correct theoretical organization or operationalization of variables. Examples are provided to illustrate each error, as are recommendations to minimize the occurrence of these errors.