Examining an Online Content General Outcome Measure

Abstract

The purpose of this study was to evaluate technical adequacy features of an online adaptation of vocabulary matching known as critical content monitoring. Validity and reliability studies were conducted with a sample of 106 students from one school in fifth-grade science content. Participants were administered 20 parallel forms of the general outcome measure over a 2-week period. Criterion-related validity correlations with a statewide accountability test ranged from .36 to .55 for the 20 parallel forms. A pooled estimate of a common correlation between the state test and the probes was found to be .45. While correlation differences were not found, statistically significant differences in probe mean scores were identified across the body of parallel forms. Student commentary regarding the online assessment process was largely positive. Alternate-form reliability correlations ranged from .21 to .73, with a median correlation of .56. Limitations and implications are addressed.

Validity of two general outcome measures of science and social studies achievement

“…Tyrimo rezultatai parodė, kad dalyko turinio įsisavinimas stipriau koreliuoja su jo pateikimu rašytine nei sakytine forma. Iš čia išsirutuliojo daugiau mokymosi turiniu grįstų vertinimų, įskaitant ir turinio vertinimo validumo testus (Mooney, McCarter, Russo, Blackwood, 2013), internetu administruojamą struktūruotą formuojamąjį vertinimą, kuris ir yra šio tyrimo objektas. Straipsnyje aprašomas mokinių turinio suvokimo įsisavinimas ir pagrindžiami du tyrimo klausimai:…”

A Comparison of Two Content Area Curriculum-Based Measurement Tools

“…Curriculum-based measurement (CBM; Deno, 1985) has been widely accepted as a valid and reliable technology for assisting educators with making data-based screening (i.e., identifying students at risk of academic difficulties) and progress decisions (i.e., measuring growth over time) in reading, mathematics, and writing (M. K. Hosp, Hosp, & Howell, 2016). CBM tools in content areas such as social studies (e.g., Beyers, Lembke, & Curs, 2013;Espin, Busch, Shin, & Kruschwitz, 2001;Mooney, McCarter, Russo, & Blackwood, 2013), and more recently science (e.g., Borsuk, 2010;Espin et al, 2013;Ford & Hosp, in press; J. L. Hosp & Ford, 2014;Mooney, Lastrapes, Marcotte, & Matthews, 2016) have also increasingly been examined.…”

Statement verification for science

“…No items representing the engineering and technology domain were identified for either Form A or B. those related to evidence of validity were less than desired. Bivariate correlations tended to be lower for both reliability and evidence of validity for SV-S than for those for other CBM tools for use in content areas (e.g., Espin et al, 2001;Espin et al, 2005;Mooney et al, 2010;Mooney et al, 2013;Johnson et al, 2013). However, results were comparable for students in the Hold Out and Validation sets in Grades 7 and 8.…”

“…However, weak, positive evidence of criterion-related validity was found when examining the relation of SV-S with the Iowa Assessments. Studies that have examined the relation between other CBM tools for content areas with high-stakes statewide tests of accountability have found evidence of criterion-related validity in the mostly moderate to strong range (e.g., Espin et al, 2001;Espin et al, 2013;Mooney et al, 2013 important to note using it at this time to assist in making screening decisions would be inappropriate. However, it is important to note student performance results suggest the possibility of practice effects.…”

“…In an extension of the research with vocabulary matching to online tools, Mooney, McCarter, Russo, and Blackwood (2013) evaluated the technical adequacy of vocabulary matching. Science students from Grade 5 (N = 106) were administered 20 parallel forms over a 2-week period.…”