In 2017, the mathematics assessments that are part of the National Assessment of Educational Progress (NAEP) program underwent a transformation shifting the administration from paper-and-pencil formats to digitally-based assessments (DBA). This shift introduced new interactive item types that bring rich process data and tremendous opportunities to study the cognitive and behavioral processes that underlie test-takers’ performances in ways that are not otherwise possible with the response data alone. In this exploratory study, we investigated the problem-solving processes and strategies applied by the nation’s fourth and eighth graders by analyzing the process data collected during their interactions with two technology-enhanced drag-and-drop items (one item for each grade) included in the first digital operational administration of the NAEP’s mathematics assessments. Results from this research revealed how test-takers who achieved different levels of accuracy on the items engaged in various cognitive and metacognitive processes (e.g., in terms of their time allocation, answer change behaviors, and problem-solving strategies), providing insights into the common mathematical misconceptions that fourth- and eighth-grade students held and the steps where they may have struggled during their solution process. Implications of the findings for educational assessment design and limitations of this research are also discussed.
Prior work on the CBAL™ mathematics competency model resulted in an initial competency model for middle school grades with several learning progressions (LPs) that elaborate central ideas in the competency model and provide a basis for connecting summative and formative assessment. In the current project, we created a competency model for Grades 3–5 that is based on both the middle school competency model and the Common Core State Standards (CCSS). We also developed an LP for rational numbers based on an extensive literature review, consultations with members of the CBAL mathematics team and other related research staff at Educational Testing Service, input from an advisory panel of external experts in mathematics education and cognitive psychology, and the use of small‐scale cognitive interviews with students and teachers. Elementary mathematical understanding, specifically that of rational numbers, is viewed as fundamental and critical to developing future knowledge and skill in middle and high school mathematics and therefore essential for success in the 21st century world. The competency model and the rational number LP serve as the conceptual basis for developing and connecting summative and formative assessment as well as professional support materials for Grades 3–5. We report here on the development process of these models and future implications for task development.
In this paper, we provide evidence of the impact of early algebra (EA) over time. We document this impact in the following ways: (a) by showing the performance over time of an experimental group of 15 children on an algebra assessment, from 3rd to 5th grade; and (b) by showing how the performance on an algebra assessment of children from an experimental group differs from the performance of a group of comparison students from their same elementary school who did not receive EA instruction from 3rd to 5th grade. We compared students’ scores through comparisons of means, correspondence factorial analyses, and hierarchical analyses. Our results highlight the positive impact of an early access to algebra, indicating that this early access is associated, when we compare 3rd graders to 5th graders, with increased scores on items that involve inequalities and graphs. When comparing experimental to comparison-group students we find increased scores on items that involve variables, functional relations, intra-mathematical contexts, tables, and algebraic expressions. The study adds to a body of literature that has been arguing for EA as well as a need to thread algebra throughout the mathematics curriculum, starting in the earliest grades.
This exploratory study investigated the behaviors and content of onscreen calculator usage by a nationally representative sample of eighth-grade students who responded to items from the 2017 National Assessment of Educational Progress mathematics assessment. Meaningful features were generated from the process data to infer whether students spontaneously used calculators for mathematical problem solving, how frequently and when they used them, and the nature of the operations performed on calculators. Sequential pattern mining was applied on sequences of calculator keystrokes to obtain patterns of operations that were representative of students’ problem-solving strategies or processes. Results indicated that higher scoring students not only were more likely to use calculators, but also used them in a more goal-driven manner than lower scoring students.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.