Teacher Trainees’ Administration and Scoring Errors on the Kaufman Test of Educational Achievement

Abstract

Achievement tests are used to make high-stakes (e.g., special education placement) decisions, and previous research on norm-referenced assessment suggests that errors are ubiquitous. In our study of 42 teacher trainees, utilizing five of the six core subtests of the Kaufman Test of Educational Achievement, Third Edition (KTEA-3), we found that while most trainees make errors, they do not make a large number per person with the exception of a few error-prone trainees. In addition, Wilcoxon signed-rank tests indicated that reading comprehension was the most prone subtest to administration ( T = 120; p < .001) and clerical ( T = 45; p < .01) errors. However, subtests pairwise comparisons indicated nonsignificant differences between error rates across subtests. Based on these findings, we recommend that training programs focus extra attention on reading comprehension and remediating students who make a disproportionate number of errors. Implications for future research are also noted.

Academic Achievement Test Use and Assessment Practices: A National Survey of Special Education Administrators

“…To achieve and maintain competency, adequate pre-service and in-service training in the areas of standardized assessment, psychometrics, and test interpretation is necessary for educators who fill these roles. This is especially true in Western states where special education teachers administer the majority of NRTs, which are difficult to administer with fidelity, and errors are ubiquitous (Lockwood et al, 2019; Styck & Walsh, 2016). Second, the limited administrations by school psychologists in rural settings suggest a continuing need for recruitment in these locales.…”

“…Additionally, this finding suggests that it may be prudent to provide additional training in rural locales where more nonschool psychologists are likely to fill this role. At a minimum, training should focus on the importance of accurate, standardized administration, common errors (see Lockwood et al, 2019), and on interpretation. Third, as a significant number of special educators and educational diagnosticians administer NRTs, future research should examine the training that these professionals receive in areas crucial for proper test administration, such as psychometrics, statistics, and standardized procedures (Lockwood & Farmer, 2019); relatedly, future research into the NRT administration fidelity of these practitioners would also appear crucial.…”

“…While the CEC acknowledges the need for assessment literacy, we were unable to find any literature describing the role that special education teachers typically fill in the administration of these measures, what measures they use, and how test administration roles may vary by state in the United States. However, Lockwood et al (2019) suggested that it is unlikely that special education teachers are tasked with engaging in norm-referenced academic achievement test (NRT) administration more than six times per year. Additionally, these authors argued that while little is known regarding the training that special education teachers receive in NRT administration, they likely do not have as extensive of training in assessment as other professionals (i.e., school psychologists).…”

“…NRTs should be administered by individuals who have training in the clinical use of the instruments in order to reduce the risk of invalid test scores and interpretation (Breaux & Lichtenberger, 2016). Current research examining the use of NRTs by nonschool psychology team members is limited (Harrison et al, 2018; Lockwood et al, 2019). Google Scholar and EBSCOhost searches revealed no studies documenting the frequency of administration, interpretation, and use of NRTs across educational professionals (e.g., teachers).…”

“…Furthermore, we wanted to determine if NRT administration roles shifted after initial eligibility (e.g., re-evaluations) to mitigate these shortages. This knowledge is important because NRTs are high-stakes and difficult to administer with fidelity (Lockwood et al, 2019; Styck & Walsh, 2016). Therefore, highly trained and practiced professionals (e.g., school psychologists) may obtain more valid test scores resulting in improved special education classification/placement decisions and service provision than special education teachers.…”