Measuring Early Childhood Mathematical Cognition: Validating and Equating Two Forms of the Research-Based Early Mathematics Assessment

Abstract

Psychometric work with young children faces the particular challenge that children’s attention spans are relatively short, and therefore, shorter assessments are required while retaining comprehensive coverage. This article reports on three empirical studies that encompass the development and validation of the research-based early mathematics assessment-short form (REMA-SF), an instrument that measures the early mathematical competency of children from 3 to 8 years of age. The developed measure captures both children’s mathematical performance and the strategies children use to solve math problems. Results indicated that the REMA-SF can produce valid scores for measuring children’s math skills in early childhood, and the validity of the measure can be well-generalized to an external (or independent) sample. Additionally, we also equated the REMA scores between the long and short forms of the assessment: anchor items common across the forms were selected and refined in the equating process.

Evaluating the Consequential Validity of the Research-Based Early Mathematics Assessment

“…The data in the present research were collected using the REMA-SF. In the previous development study of the REMA-SF (Dong et al, 2021), multiple sources of validity evidence were established, such as sufficient content coverage, clear unidimensionality, good model-data-fit indices, as well as satisfying reliability and separation. Moreover, in the same investigation, this evidence was well-replicated to an external sample, which demonstrates that the validity of the REMA-SF is generalizable (i.e., cross-observer validity).…”

“…We found a person reliability of .93 and a separation of 3.58 in this sample, which indicates that the REMA-SF scores were highly reliable (Wright & Masters, 1982). The strategy items allow a better differentiation among students with various levels of math competency (Clements et al, 2008b; Dong et al, 2021), and provide more effective and specific feedback regarding children’s learning to stakeholders (e.g., teachers) compared to item correctness only. Such cognitive-process feedback derived from children’s test scores can be perceived as major evidence for consequential validity (Iliescu & Greiff, 2021), because it may positively contribute to instruction and children’s future learning outcomes (e.g., amplify the effect of differentiated instruction or set more accurate learning goals for each child).…”

“…In response to this critical need, Clements and colleagues (Clements et al, 2008(Clements et al, /2019a have conducted empirical studies to develop and validate the Research-based Early Mathematics Assessment (REMA), an instrument that measures the early mathematical competency of children from 3 to 8 years of age. In addition to the original full version, a short-form version of the REMA (i.e., REMA-SF) has been developed and validated to have a shorter assessment time and a higher level of efficiency during its administration (Dong et al, 2021).…”

“…Consequential validity, or the degree of score uses and interpretations that can support justified and fair decisions (Messick, 1995), has been recognized as an essential aspect of testing practices by many researchers and psychological professionals (e.g., AERA/APA/NCME, 2014; Dumas et al, 2022; Iliescu & Greiff, 2021). The REMA is a major measurement tool in early childhood research, and previous studies have demonstrated its internal and external validity using a variety of methods (Clements et al, 2008b; Dong et al, 2021). However, the consequential validity and fairness of the REMA has not been comprehensively investigated.…”

Psychometric Evaluation of the Preschool Early Numeracy Skills Test–Brief Version Within the Item Response Theory Framework

“…Several measures have been developed for the assessment of either discrete numeracy skills (e.g., Floyd et al., 2006; Hellstrand et al., 2020; Van de Rijt et al., 1999) or broad mathematical ability (e.g., Clements et al., 2008; Ginsburg & Baroody, 2003; Starkey et al., 2004). Apart from their strong psychometric characteristics, mathematical measures for early childhood instruments should also follow a developmentally appropriate administration (Clements & Sarama, 2016; Dong et al., 2021). However, several existing measures have been criticized for being complicated to administer (demanding additional materials like cards, blocks, tokens) and time‐consuming (Hellstrand et al., 2020; Purpura & Lonigan, 2015).…”