Applying model analysis to a resource-based analysis of the Force and Motion Conceptual Evaluation

Smith, Trevor I.; Wittmann, Michael; Carter, Thomas

doi:10.1103/physrevstper.10.020102

Cited by 30 publications

(41 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The three distinct clusters described by Thornton et al have been studied at the item level [51,52]. In 2008, Smith and Wittmann introduced revised clusters and investigated student response patterns on those sets of items.…”

Section: Fmcementioning

confidence: 99%

See 1 more Smart Citation

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Henderson

Miller

Stewart

et al. 2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Gender gaps on the most widely used conceptual inventories created by physics education researchers have been extensively studied. Most of the research exploring the consistent gender gaps has been performed at the student level using the total evaluation score; less research has been performed examining these assessments at the item level and this research has been predominately restricted to the Force Concept Inventory (FCI). Many studies have identified subsets of FCI items as unfair to either men or women. An item is fair if men and women of equal ability in conceptual physics score equally on the item. This study explored the item-level gender fairness of the Force and Motion Conceptual Evaluation (FMCE) and the Conceptual Survey of Electricity and Magnetism (CSEM). Classical test theory and differential item functioning (DIF) analysis were employed to examine item fairness. Fairness was investigated with four large post-test samples, two for the FMCE (n 1 ¼ 3016, n 2 ¼ 3719) and two for the CSEM (n 1 ¼ 2014, n 2 ¼ 2657). Men and women performed significantly differently on the majority of FMCE items but with no more than a small effect size. There were fewer items in the CSEM where men and women performed differently. Using DIF analysis, which assumes that overall test score is an accurate measure of ability, only one item in the FMCE demonstrated large DIF in both samples with that item unfair to women. One additional item showed large DIF in a single sample, also unfair to women. Only one item in the CSEM demonstrated large DIF. The item was unfair to men but this result was not consistent across all samples. The number of large DIF items identified in both the FMCE and the CSEM was substantially smaller than the number of large DIF items identified in the FCI by previous studies.

show abstract

Section: Fmcementioning

confidence: 99%

“…They also introduced cluster 40_43 as velocity graphs [51]. In 2014, Smith, Wittmann, and Carter used these revised clusters to provide insight into the effectiveness of interactive classroom techniques [52].…”

Section: Fmcementioning

confidence: 99%

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Henderson

Miller

Stewart

et al. 2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

show abstract

“…"Acceleration Graph" items (items [22][23][24][25][26] ask students to select the graph which correctly represents the acceleration of a toy car moving on a horizontal surface. "Coin Toss -Acceleration" items (items [27][28][29] ask students to select the acceleration of a coin tossed in the air. "Newton III" items (items [30][31][32][33][34][35][36][37][38][39] ask students about the forces during a variety of interactions between cars and trucks.…”

Section: A the Fmce Instrumentmentioning

confidence: 99%

Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis

Wells

Henderson

Traxler

et al. 2020

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Investigating student learning and understanding of conceptual physics is a primary research area within Physics Education Research (PER). Multiple quantitative methods have been employed to analyze commonly used mechanics conceptual inventories: the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE). Recently, researchers have applied network analytic techniques to explore the structure of the incorrect responses to the FCI identifying communities of incorrect responses which could be mapped on to common misconceptions. In this study, the method used to analyze the FCI, Modified Module Analysis (MMA), was applied to a large sample of FMCE pretest and post-test responses (Npre = 3956, Npost = 3719). The communities of incorrect responses identified were consistent with the item groups described in previous works. As in the work with the FCI, the network was simplified by only retaining nodes selected by a substantial number of students. Retaining nodes selected by 20% of the students produced communities associated with only four misconceptions. The incorrect response communities identified for men and women were substantially different, as was the change in these communities from pretest to post-test. The 20% threshold was far more restrictive than the 4% threshold applied to the FCI in the prior work which generated similar structures. Retaining nodes selected by 5% or 10% of students generated a large number of complex communities. The communities identified at the 10% threshold were generally associated with common misconceptions producing a far richer set of incorrect communities than the FCI; this may indicate that the FMCE is a superior instrument for characterizing the breadth of student misconceptions about Newtonian mechanics. *

show abstract

“…Smith, Wittmann, and Carter found that learning trends differed on the Force Sled question cluster (in which answer choices are presented as descriptions of forces) and the Force Graphs cluster (answers as graphs of force vs. time) [2]; however, the cause of these differences is yet unknown. Do students struggle to interpret graphs of forces as is well documented for graphs of kinematic quantities [3,4]?…”

Section: Introductionmentioning

confidence: 99%

“…We are particularly interested in documenting cases in which students have a relatively high tendency to exist in a superposition or mixed-model state (selecting incoherent responses within a case) [6], and identifying possible hierarchies in student learning for various questions [7]. To highlight various aspects of our results we present examples from three different schools, two of which were previously analyzed [2]. All courses used research-validated instructional materials.…”

mentioning

confidence: 99%

A Multi-faceted Approach to Measuring Student Understanding

Griffin¹,

Louis²,

Moyer³

et al. 2016

2016 Physics Education Research Conference Proceedings

Self Cite

View full text Add to dashboard Cite

Data from the FMCE suggest that there may be inconsistencies in students' understanding of forces when various types of motion are presented. These inconsistencies are highly evident when testing for students' understanding using graphs. In the current study we assume that measurements of student understanding of a particular topic depend both on the student and on the instrument used to make the measurement. Multiple measurements are needed to build a more complete picture of what the student believes to be true. We compare individual student responses to 12 questions from three FMCE question clusters. Using various visualizations of the data including model analysis plots, contingency tables of student responses, and consistency plots, we identify trends that are not evident from traditional normalized-gain-based analyses. Statistical results from the χ 2 test of independence and one-way analysis of variance provide support for our findings.

show abstract

Applying model analysis to a resource-based analysis of the Force and Motion Conceptual Evaluation

Cited by 30 publications

References 17 publications

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis

A Multi-faceted Approach to Measuring Student Understanding

Contact Info

Product

Resources

About