A primer on standardized testing: History, measurement, classical test theory, item response theory, and equating

Self Cite

Objective: The objectives of this study were to (1) identify factors predictive of performance on the National Board of Chiropractic Examiners Part IV exam and (2) investigate correlations between the scores obtained in the Part I, Part II, Physiotherapy, and Part III exams and the Part IV examination. Methods: A random sample of 1341 records was drawn from National Board of Chiropractic Examiners data to investigate the relationships between the scores obtained on the National Board of Chiropractic Examiners exams. A hierarchical multiple regression analysis related the performance on Part IV to examinee's gender, Part IV repeater status, and scores obtained on the Part I, Part II, Physiotherapy, and Part III exams. Results: The analyses revealed statistical relations among all National Board of Chiropractic Examiners exams. The correlations between Part IV and Part I ranged from r = .31 to r = .4; between Part IV and Part II from r = .34 to r = .45. The correlation between Part IV and Physiotherapy was r = .44; between Part IV and Part III was r = .46. The strongest predictors of the Part IV score were found to be examinees' scores in Diagnostic Imaging, β̂ = .19, p < .001; Chiropractic Practice, β̂ = .17, p < .001; Physiotherapy, β̂ = .15, p < .001; and the Part III exam β̂ = .19, p < .001. Conclusions: Performance on the National Board of Chiropractic Examiners Part IV examination is related to the performance in all other National Board of Chiropractic Examiners exams.

Section: Methodsmentioning

confidence: 99%

Variables associated with successful performance on the National Board of Chiropractic Examiners Part IV examination

Himelfarb¹,

Shotts²,

Hyland³

et al. 2020

Self Cite

“…We employed methodologies based on CTT and IRT. 20,21 We also studied the decision accuracy of the redesigned DIM exam. One of the changes made to the exam was to substitute x-ray films on view boxes with digital images.…”

Section: Study Objectivesmentioning

confidence: 99%

“…For items scored dichotomously, when each answer is scored as correct or incorrect, a plethora of logistic IRT models is available. 20,21 However, when the response is scored on the ordinal scale, 30 more universal models are available for use. 31 Various polytomous IRT models had been developed to fit ordered categorical responses.…”

Section: Overview Of Statistical Modelingmentioning

confidence: 99%

The transition to digital presentation of the diagnostic imaging domain of the Part IV examination of the National Board of Chiropractic Examiners

Himelfarb¹,

Seron²,

Hyland³

et al. 2020

Self Cite

Objective: This article introduces changes made to the diagnostic imaging (DIM) domain of the Part IV of the National Board of Chiropractic Examiners examination and evaluates the effects of these changes in terms of item functioning and examinee performance. Methods: To evaluate item function, classical test theory and item response theory (IRT) methods were employed. Classical statistics were used for the assessment of item difficulty and the relation to the total test score. Item difficulties along with item discrimination were calculated using IRT. We also studied the decision accuracy of the redesigned DIM domain. Results: The diagnostic item analysis revealed similarity in item function across test forms and across administrations. The IRT models found a reasonable fit to the data. The averages of the IRT parameters were similar across test forms and across administrations. The classification of test takers into ability (theta) categories was consistent across groups (both norming and all examinees), across all test forms, and across administrations. Conclusion: This research signifies a first step in the evaluation of the transition to digital DIM high-stakes assessments. We hope that this study will spur further research into evaluations of the ability to interpret radiographic images. In addition, we hope that the results prove to be useful for chiropractic faculty, chiropractic students, and the users of Part IV scores.

“…The precision of an estimate is inversely related to the degree of sampling error. 14 The topics of score precision, validity, and reliability, as well as the efforts NBCE makes to ensure them, are discussed in Himelfarb 15 and Himelfarb et al 16…”

Section: What Is a Test?mentioning

confidence: 99%

“…Thus, the numbers provided to test takers, chiropractic institutions, and state licensing boards are calculated based on more realistic assumptions and are more precise. Himelfarb 15 and Himelfarb et al 16 provide an in-depth discussion of the differences between the 2 theories. For example, Himelfarb et al 20 explain how Part IV is scored with IRT models using the diagnostic imaging portion of the exam.…”

Section: Classical Test Theory Vs Item Response Theorymentioning

confidence: 99%

Current state and future directions of the National Board of Chiropractic Examiners

Ouzts¹,

Himelfarb²,

Shotts³

et al. 2020

Self Cite

The objective of this paper is to describe changes made to chiropractic national board examinations in the United States, including methodologies in test scoring, and to discuss future directions in test development and administration being considered by the National Board of Chiropractic Examiners (NBCE). Additionally, this paper serves as an introduction to the articles written by the NBCE staff and published in this issue of the journal. Statistical perspective on the properties of a test are presented, and reasons for the NBCE moving to item response theory for test scoring are described. NBCE consideration of on-demand testing and changes implemented in the Part IV practical examination are also discussed.