Practitioners typically face situations in which examinees have not responded to all test items. This study investigated the effect on an examinee's ability estimate when an examinee is presented an item, has ample time to answer, but decides not to respond to the item. Three approaches to ability estimation (biweight estimation, expected a posteriori, and maximum likelihood estimation) were examined. A Monte Carlo study was performed and the effect of different levels of omissions on the simulee's ability estimates was determined. Results showed that the worst estimation occurred when omits were treated as incorrect. In contrast, substitution of 0.5 for omitted responses resulted in ability estimates that were almost as accurate as those using complete data. Implications for practitioners are discussed.
The purpose of the present research was to develop general guidelines to assist practitioners in setting up operational computerized adaptive testing (CAT) systems based on the graded response model. Simulated data were used to investigate the effects of systematic manipulation of various aspects of the CAT procedures for the model. The effects of three major variables were examined: item pool size, the stepsize used along the trait continuum until maximum likelihood estimation could be calculated, and the stopping rule employed. The findings suggest three guidelines for graded response CAT procedures: (1) item pools with as few as 30 items may be adequate for CAT; (2) the variable-stepsize method is more useful than the fixedstepsize methods; and (3) the minimum-standard-error stopping rule will yield fewer cases of nonconvergence, administer fewer items, and produce higher correlations of CAT &thetas; estimates with full-scale estimates and the known &thetas;s than the minimum-information stopping rule. The implications of these findings for psychological assessment are discussed. Index terms: computerized adaptive testing, graded response model, item response theory, polychotomous scoring.
This study examined the factorial validity of scores on the newly developed Students' Evaluation of Teaching Effectiveness Rating Scale (SETERS) through a series of confirmatory and multilevel structures. Conventional confirmatory factor analyses using the total covariance and pooled within-covariance matrices from two midwestern universities indicated that a reduced 25-item SETERS fit the data better than the original 34-item SETERS. Furthermore, multilevel factor analysis was conducted on the combined samples. This analysis suggested that one or three factors at the between and within levels were a plausible representation of SETERS scores. Pearson's correlations between individual scores on the SETERS and the Students' Evaluation of Educational Quality questionnaire provided additional validity evidence for the two measures. The need for additional empirical research on the SETERS before widespread use is discussed.
Anxiety is one of the most pervasive symptoms seen in clinical psychological disorders. The Beck Anxiety Inventory (BAI) is a popular measure to assess the construct of anxiety. This study was concerned with an examination of potential study factors that are associated with the variability of the reliability estimates of the BAI scores. A review of the literature involving the BAI showed that more than 57% of the publications either did not mention reliability estimates for BAI scores or presented secondary reliability estimates. This meta-analysis of coefficient alpha and test-retest reliability estimates showed that diagnostic classification of participants and the within-study BAI score variability were related to the magnitude of the reliability estimates.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.