Eye-tracking of visual attention in web-based assessment using the Force Concept Inventory

Han, Jing; Chen, Li; Fu, Zhao; Fritchman, Joseph C.; Bao, Lei

doi:10.1088/1361-6404/aa6c49

Cited by 23 publications

(12 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Even in the highest levels of expertise, the most popular incorrect options, that were designed to address popular misconceptions and learning difficulties with graphs, receive more attention that the other incorrect options. Expert students (as indicated by high test scores) who shifted their attention to the correct choices still keep a higher level of attention to the popular options, indicating conceptual mixing [20]. From an assessment point of view, this result provides evidence for the test validity at the behavioural level.…”

Section: Discussionmentioning

confidence: 64%

Visual attention while solving the test of understanding graphs in kinematics: an eye-tracking analysis

et al. 2020

View full text Add to dashboard Cite

This study used eye-tracking to capture the students' visual attention while taking the test of understanding graphs in kinematics (TUG-K). A total of N = 115 upper-secondary-level students from Germany and Switzerland took the 26-item multiple-choice instrument after learning about kinematics graphs in the regular classroom. Besides choosing the correct alternative among research-based distractors, the students were required to judge their response confidence for each question. The items were presented sequentially on a computer screen equipped with a remote eye tracker, resulting in a set of approx. 3000 paired responses (accuracy and confidence) and about 40 hours of eye movementdata (approx. 500.000 fixations). The analysis of students' visual attention related to the item stems (questions) and the item options reveal that high response confidence is correlated with shorter visit duration on both elements of the items. While the students' response accuracy and their response confidence are highly correlated on the score level, r(115) = 0.63, p < 0.001, the eye-tracking measures do not sufficiently discriminate between correct and incorrect responses. However, a more fine-grained analysis of visual attention based on different answer options reveals a significant discrimination between correct and incorrect answers in terms of an interaction effect: Incorrect responses are associated with longer visit durations on strong distractors and less time spent on correct options while correct responses show the opposite trend. Outcomes of this study provide new insights into the validation of concept inventories based on students' behavioural level.

show abstract

Section: Discussionmentioning

confidence: 64%

Visual attention while solving the test of understanding graphs in kinematics: an eye-tracking analysis

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Recent studies have (i) explored students' visual attention in different areas of interest (AOI) to distinguish between experts and novices or between good and poor performances [32][33][34], (ii) used transition analysis to investigate how students switch back and forth between different AOIs while learning or problem solving to get insight into student expertise or the characteristics of the learning material [35][36][37][38], (iii) used classical eye-tracking measures as fixation durations or the number of fixations to predict learning outcomes [39], and (iv) touched on problems and issues peripherally related to PER, such as the troubleshooting of malfunctioning circuits [40], comprehending malfunctioning mechanical devices or how mechanical systems work [41,42], and how spatial ability influences solving kinematics problems with trajectories [43].…”

Section: B Eye Tracking and Permentioning

confidence: 99%

Instruction-based clinical eye-tracking study on the visual interpretation of divergence: How do students look at vector field plots?

Klein

Viiri

Mozaffari

et al. 2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Relating mathematical concepts to graphical representations is a challenging task for students. In this paper, we introduce two visual strategies to qualitatively interpret the divergence of graphical vector field representations. One strategy is based on the graphical interpretation of partial derivatives, while the other is based on the flux concept. We test the effectiveness of both strategies in an instruction-based eye-tracking study with N ¼ 41 physics majors. We found that students' performance improved when both strategies were introduced (74% correct) instead of only one strategy (64% correct), and students performed best when they were free to choose between the two strategies (88% correct). This finding supports the idea of introducing multiple representations of a physical concept to foster student understanding. Relevant eye-tracking measures demonstrate that both strategies imply different visual processing of the vector field plots, therefore reflecting conceptual differences between the strategies. Advanced analysis methods further reveal significant differences in eye movements between the best and worst performing students. For instance, the best students performed predominantly horizontal and vertical saccades, indicating correct interpretation of partial derivatives. They also focused on smaller regions when they balanced positive and negative flux. This mixedmethod research leads to new insights into student visual processing of vector field representations, highlights the advantages and limitations of eye-tracking methodologies in this context, and discusses implications for teaching and for future research. The introduction of saccadic direction analysis expands traditional methods, and shows the potential to discover new insights into student understanding and learning difficulties.

show abstract

“…In addition to students' scores and explanations on problems with graphs that we analyzed in the previous studies [10,11,14], we used eye tracking in this study to investigate where students allocate visual attention during problem solving. Measurement of eye movements is an increasingly used method in PER [37][38][39][40][41][42][43][44][45][46][47][48][49]. There are a number of eye-tracking studies on understanding of graphs [36,39,46,[48][49][50][51].…”

Section: Introductionmentioning

confidence: 99%

Student understanding of graph slope and area under a graph: A comparison of physics and nonphysics students

Sušac

Bubić

Kazotti

et al. 2018

Phys. Rev. Phys. Educ. Res.

View full text Add to dashboard Cite

Previous studies have identified physics students' difficulties with graph slope and area under a graph in different contexts. In this study we compared physics and nonphysics (psychology) students' understanding of graphs; i.e., we evaluated the effects of concept (graph slope vs area under graph), type of question (qualitative vs quantitative), and context (physics vs finance) on their scores, strategies, and eye-tracking data. All students solved questions about graph slope better than the questions about area under a graph. Psychology students scored rather low on the questions about area under a graph, and physics students spent more time on questions about area under a graph than on slope questions, indicating that understanding of area under a graph is quite a difficult concept that seems unlikely to develop spontaneously. Generally, physics students had comparable scores on the qualitative and quantitative questions, whereas psychology students solved qualitative questions much better. As expected, students' scores and eye-tracking measures indicated that problems involving physics context were easier for physics students since they typically had higher scores and shorter total and axes viewing times for physics than finance questions. Some physics students may have transferred the concepts and techniques from physics to finance because they typically scored better than psychology students on the finance questions that were novel for both groups. Analysis of student strategies showed that physics students mostly relied on strategies learned in physics courses, with strong emphasis on the use of formulas, whereas psychology students mostly used common-sense strategies, as they did not know the physics formulas. The implications of the results for teaching and learning about graphs in physics courses are also discussed.

show abstract

Eye-tracking of visual attention in web-based assessment using the Force Concept Inventory

Cited by 23 publications

References 29 publications

Visual attention while solving the test of understanding graphs in kinematics: an eye-tracking analysis

Visual attention while solving the test of understanding graphs in kinematics: an eye-tracking analysis

Instruction-based clinical eye-tracking study on the visual interpretation of divergence: How do students look at vector field plots?

Student understanding of graph slope and area under a graph: A comparison of physics and nonphysics students

Contact Info

Product

Resources

About