We examined normalized gains and preinstruction scores on the force concept inventory ͑FCI͒ for students in interactive engagement courses in introductory mechanics at four universities and found a significant, positive correlation for three of them. We also examined class average FCI scores of 2948 students in 38 interactive engagement classes, 31 of which were from the same four universities and 7 of which came from 3 other schools. We found a significant, positive correlation between class average normalized FCI gains and class average preinstruction scores. To probe this correlation, we administered Lawson's classroom test of scientific reasoning to 65 students and found a significant, positive correlation between these students' normalized FCI gains and their Lawson test scores. This correlation is even stronger than the correlation between FCI gains and preinstruction FCI scores. Our study demonstrates that differences in student populations are important when comparing normalized gains in different interactive engagement classes. We suggest using the Lawson test along with the FCI to measure the effectiveness of alternative interactive engagement strategies.

Preinstruction SAT scores and normalized gains ͑G͒ on the force concept inventory ͑FCI͒ were examined for individual students in interactive engagement ͑IE͒ courses in introductory mechanics at one high school ͑N = 335͒ and one university ͑N = 292͒, and strong, positive correlations were found for both populations ͑r = 0.57 and r = 0.46, respectively͒. These correlations are likely due to the importance of cognitive skills and abstract reasoning in learning physics. The larger correlation coefficient for the high school population may be a result of the much shorter time interval between taking the SAT and studying mechanics, because the SAT may provide a more current measure of abilities when high school students begin the study of mechanics than it does for college students, who begin mechanics years after the test is taken. In prior research a strong correlation between FCI G and scores on Lawson's Classroom Test of Scientific Reasoning for students from the same two schools was observed. Our results suggest that, when interpreting class average normalized FCI gains and comparing different classes, it is important to take into account the variation of students' cognitive skills, as measured either by the SAT or by Lawson's test. While Lawson's test is not commonly given to students in most introductory mechanics courses, SAT scores provide a readily available alternative means of taking account of students' reasoning abilities. Knowing the students' cognitive level before instruction also allows one to alter instruction or to use an intervention designed to improve students' cognitive level.

Articles you may be interested inLearning of content knowledge and development of scientific reasoning ability: A cross culture comparison Am.Abstract. We observe no significant effect of gender on grades in our IE introductory mechanics courses at Loyola Marymount University, but we do observe a significant gender gap on FCI normalized gains, with males achieving higher gains than females. Over the past three years, FCI gains have improved for both male and female students in IE classes taught with the Thinking in Physics (TIP) pedagogy. However, a gender gap on FCI gains remains, even when scientific reasoning abilities are taken into account. Indeed, the gap appears much greater for students with the strongest scientific reasoning skills and the highest FCI gains. Data from IE introductory physics courses using modeling at Edward Little High School in Maine show a similar result with some additional data showing a reverse gender gap for those students with very weak scientific reasoning skills.

Many teachers administer a force concept test such as the Force Concept Inventory1,2 (FCI) to their students in an effort to evaluate and improve their instructional practices. It has been commonly assumed that looking at class normalized gains allows teachers to compare their courses with other courses. In this paper we present evidence to suggest that the use of class normalized gains alone may not provide a complete picture. We argue that student reasoning ability should also be assessed before between-course comparisons can be made. Assessment of reasoning ability is also useful in identifying students who are at risk. In the following we shall concentrate on the FCI, but we think our conclusions probably apply to physics concept tests generally.

Recently, Nissen et al. argued in this journal for the use of Cohen's d, in place of the more commonly used normalized gain, in the analysis of preinstruction and postinstruction scores on concept inventories used to measure the effectiveness of instruction. Their reason for advocating such a change is that they say normalized gains are "prescore biased." We provide five examples, including one cited by Nissen, that show no prescore bias when data are carefully analyzed, demonstrating that the problem with their analysis is omitted variable bias. We show that Cohen's d is less informative than normalized gain when used as a single parameter measure of teaching effectiveness, even though, as Nissen points out, d is more widely used in other fields. We believe that physics education researchers should continue to use normalized gain to assess educational effectiveness of pedagogy. However, because different student populations can have significantly different responses to the same pedagogy, in any interpretation of normalized gain, it is important to consider a measure of the abilities of the students. In analyzing normalized gains for the Force Concept Inventory (FCI), average scores on either Lawson's Test of Scientific Reasoning Ability or the SAT should be considered, because these scores are strongly correlated with normalized gain, indicating student abilities may have a greater impact on the gains achieved in a class than the specific pedagogy used.