A Survey on the Usage of Eye-Tracking in Computer Programming

Abstract

Traditional quantitative research methods of data collection in programming, such as questionnaires and interviews, are the most common approaches for researchers in this field. However, in recent years, eye-tracking has been on the rise as a new method of collecting evidence of visual attention and the cognitive process of programmers. Eyetracking has been used by researchers in the field of programming to analyze and understand a variety of tasks such as comprehension and debugging. In this paper, we will focus on reporting how experiments that used eye-trackers in programming research are conducted, and the information that can be collected from these experiments. In this mapping study, we identify and report on 63 studies, published between 1990 and June 2017, collected and gathered via manual search on digital libraries and databases related to computer science and computer engineering. Among the five main areas of research interest are program comprehension and debugging, which received an increased interest in recent years, non-code comprehension, collaborative programming and requirements traceability research, which had the fewest number of publications due to possible limitations of the eye-tracking technology in this type of experiments. We find that most of the participants in these studies were students and faculty members from institutions of higher learning, and while they performed programming tasks on a range of programming languages and programming representations, we find Java language and UML representation to be the most used materials. We also report on a range of eye-trackers and attention tracking tools that have been utilized, and find Tobii eye-trackers to be the most used devices by researchers.

Visually Analyzing Students' Gaze on C++ Code Snippets

“…It is a precursor to many software engineering activities such as bug fixing, code reviewing, or new feature additions. Since 1990 and more so after 2006, eye tracking researchers in software engineering have been working towards conducting studies to build on the body of knowledge of program comprehension in a systematic way [2], [3]. We only expect this trend to continue as eye trackers get more accessible and are more suitable for the community to adopt as support infrastructure [4] improves.…”

Developer Reading Behavior While Summarizing Java Methods: Size and Context Matters

“…We analyze eye-tracking data based on method size using the same approach in RQ1 by adjusting the gaze time and number of visits for each location (signature, body, calls, control flow). Then, methods are divided by their size -large (40-80 LOC), medium (23)(24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36)(37)(38)(39), and small (9-22 LOC). For each category, the eye movements on signature, call, and control flow statements are analyzed.…”

“…Eye Tracking Studies: Eye-tracking technology is being used in software engineering [20][21][22][23][24][25] to study how programmers read [4,[26][27][28][29][30], review [31][32][33], and summarize [17,28,34] source code. Crosby et al [4] conducted an early eye-tracking study of high and low experience programmers reading a binary search algorithm.…”

Early childhood preservice teachers’ debugging block-based programs: An eye tracking study

“…Research that uses eye-tracking devices to assess learning in computer science has increased in recent years. A survey on the use of eye-tracking in programming instruction research revealed that program comprehension and debugging are two mostly studied areas (Obaidellah, Al Haek, and Cheng, 2018). Among these studies, several have used eye movement data to assess "learners' problem-solving processes objectively" (Sun and Hsu, 2019, p. 67) as learners attempt to understand and/or debug programs.…”

“…These studies demonstrate the potential of using eye trackers to understand learners' cognitive processes during complex programming tasks. However, most research has been conducted with higher education individuals from the field of computer science, and involved text-based programming languages (Obaidellah et al, 2018). To our knowledge, the present study is the first attempt to use eye tracking to understand preservice teachers' cognitive processes during debugging block-based programs.…”