Interpreting the force concept inventory: A response to March 1995 critique by Huffman and Heller

Abstract

The Force Concept Inventory (FCI) 1 is a unique kind of "test" designed to assess student understanding of the most basic concepts in Newtonian physics. It can be used for several different purposes, but the most important one is to evaluate the effectiveness of instruction.For that purpose, the FCI is probably the most widely used instrument in physics education today. Results of many independent investigations have been reported at the biannual AAPT meetings since the FCI was published in March 1992. Including unpublished data that have been brought to our attention, we estimate that the FCI has been administered in classes of well over a hundred different teachers to more than ten thousand students in high schools, colleges and universities. For comparative analysis, Richard Hake 2 has been collecting data on university and high school physics taught by many different teachers and methods.Readers with a similar concern are urged to contact him. Douglas Huffman and Patricia Heller (H&H) 3 have recently published a "factor analysis" of FCI data and claimed that it raises serious concerns about the validity and interpretation of the FCI. We find that their data provide support for our position, but their concerns are unjustified. Moreover, they have overlooked relevant analysis of the issues in our published papers, and their advice on using the FCI is ill considered. In the following we take familiarity with references 1 and 3 for granted, and readers who have not studied them are encouraged to do so. Design and Development of the FCIThe FCI has a predecessor, the Mechanics Diagnostic Test (MDT) 4 , which has also been widely applied. About 60% of the FCI is the same as the MDT, and the results from both tests are perfectly consistent and mutually supportive. Analysis of MDT results led to the improvements in the FCI. Accordingly, we regard the FCI as an improved version of the MDT rather than a completely new test. We mention this because the data and analysis in our two papers on MDT 4,5 have strong bearing on the interpretation of the FCI and its results.

Development and application of a four-tier test to assess pre-service physics teachers’ misconceptions about geometrical optics

“…It is expected that these values should be minimized for establishing construct validity (Hestenes & Halloun, 1995).…”

“…Students below this threshold were thought to have the following characteristics: undifferentiated concepts of velocity and acceleration, lack of a vectorial concept of velocity; lack of a universal force concept; fragmented and incoherent concepts about force and motion. An FCI score of 85 %, on the other hand, was interpreted as the Newtonian Mastery threshold, and anyone who gets above this score is considered as confirmed Newtonian thinker (Hestenes & Halloun, 1995 high school and 750 university students that the FCI actually did not measure the six conceptual dimensions that were initially mentioned as comprising a force concept. For high school students, a factor analysis produced a total of ten factors, but only two of them accounted for enough variance to be considered significant.…”

“…False Positive: A correct response that is chosen by individuals who do not have the correct, scientific reasoning for the question Hestenes & Halloun, 1995).…”

Separating the Biased Perceptions from Lack of Knowledge on Entrepreneurship Through a Diagnostic Instrument: LEDI

“…Content validities are established based on expert opinions and the percentages of false positives and false negatives. According to Hestenes and Halloun (1995), if items of the test are clear and understandable, the knowledgeable participants would answer the test more correctly than the unknowledgeable ones. When these values are minimized, multiple-choice tests have greater validity.…”

“…False positive is an effect which is not actually present and false negative is failing to reveal an effect that is actually present. Hestenes and Halloun (1995) recommend that a correct answer along with a wrong reason (false positive), and a wrong answer followed by a correct reason (false negative) can be used to provide evidence for content validity in the development of diagnostic test. They state that minimization of these probabilities provide high validity.…”