How readability and topic incidence relate to performance on mathematics story problems in computer-based curricula.

Abstract

Solving mathematics story problems requires text comprehension skills. However, previous studies have found few connections between traditional measures of text readability and performance on story problems. We hypothesized that recently developed measures of readability and topic incidence measured by text-mining tools may illuminate associations between text difficulty and problem-solving measures. We used data from 3,216 middle and high school students from 10 schools using the Cognitive Tutor Algebra program; these schools were geographically, socioeconomically, racially, and ethnically diverse. We found that several indicators of the readability and topic of story problems were associated with students' tendency to give correct answers and request hints in Cognitive Tutor. We further examined the individual skill of writing an algebraic expression from a story scenario, and examined students at the lowest performing schools in the sample only, and found additional associations for these subsets. Key readability and topic categories that were related to problem-solving measures included word difficulty, text length, pronoun use, sentence similarity, and topic familiarity. These findings are discussed in the context of models of mathematics story problem solving and previous research on text comprehension.

How Readability Factors Are Differentially Associated With Performance for Students of Different Backgrounds When Solving Mathematics Word Problems

“…In prior work (Walkington, Clinton, Ritter, et al, 2015), we examined 151 story problems from a widely used Algebra I curriculum, Cognitive Tutor Algebra , as well as a set of 60 algebra story problems from a middle school tutoring software, MATHia . We entered the text of the story problems into two text analysis programs—Coh-Metrix and LIWC.…”

“…We also examined performance on the fourth- and eighth-grade released items on the NAEP and TIMSS (Walkington, Clinton, Shivraj, & Yovanoff, 2015). These data sets simply gave an overall accuracy statistic for each problem averaged for every student who took the test.…”

The Influence of a Mathematics Problem-Solving Training System on First-Year Middle School Students

“…Boonen, van Wesel, Jolles, and van der Schoot (2014) indicated that conceptual understanding through reading the context of mathematical word problems plays an import role in problem-solving. However, one of the problem-solving skills children have been found to have difficulties with is the ability to comprehend the conception of a mathematical word problem (Schoppek & Tulis, 2010;Walkington, Clinton, Ritter, & Nathan, 2015). As there is a strong relationship between reading comprehension and solving mathematical word problems, how to improve these skills while learning mathematical skills has become an important issue in achieving effective, meaningful learning through problem-solving (Lee & Chen, 2009).…”

“…Many researchers have indicated that employing CAAF to provide personalized teaching and remedial materials is essential to improving students' learning behaviors (de-Marcos, Garcia-Lopez, & Garcia-Cabot, 2016;Hsiao et al, 2014) and academic achievement in different disciplines (Hsiao et al, 2015;Hsiao, Chang, Lin, & Wang, 2016;Walkington et al, 2015;Yung & Paas, 2015). From a practical point of view, the mathematical ability of each student may contribute to the interpretation of individual differences in problem-solving.…”

Using correlational topic modeling for automated topic identification in intelligent tutoring systems

“…Teachers and researchers can use information about the underlying skill model to determine if skills differ in important ways, such as determining which skills are more likely to be associated with disengagement (e.g. [1]) or negative affect [11], and to study how hint requests and other metacognitive behaviors vary between different skills [24]. Skill models can also help to inform teachers' assessments of student learning and performance, and identify areas where students may need additional practice or scaffolding in order to succeed.…”