A practical guide on conducting eye tracking studies in software engineering

Sharafi, Zohreh; Sharif, Bonita; Guéhéneuc, Yann‐Gaël; Begel, Andrew; Bednarik, Roman; Crosby, Martha E.

doi:10.1007/s10664-020-09829-4

Cited by 66 publications

(30 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The pupil widens as a response to low light conditions [52]. Studies showed that it also happens during complex cognitive tasks [23,44,53]. Wahn et al conducted an experiment showing that the pupil size scales with attentional load and task experience, stating that the pupil dilates more the less experienced the subject is in the given task [64].…”

Section: Answer Timementioning

confidence: 99%

Influence of diagram layout and scrolling on understandability of BPMN processes: an eye tracking experiment with BPMN diagrams

2021

View full text Add to dashboard Cite

Business process modeling is an important activity for developing software systems—especially within digitization projects and when realizing digital business models. Specifying requirements and building executable workflows is often done by using BPMN 2.0 process models. Although there are several style guides available for BPMN, e.g., by Silver and Richard (BPMN method and style, vol 2, Cody-Cassidy Press, Aptos, 2009), there has not been much empirical research done into the consequences of the diagram layout. In particular, layouts that require scrolling have not been investigated yet. The aim of this research is to establish layout guidelines for business process modeling that help business process modelers to create more understandable business process diagrams. For establishing benefits and penalties of different layouts, a controlled eye tracking experiment was conducted, in which data of 21 professional software developers was used. Our results show that horizontal layouts are less demanding and that as many diagram elements as possible should be put on the initially visible screen area because such diagram elements are viewed more often and longer. Additionally, diagram elements related to the reader’s task are read more often than those not relevant to the task. BPMN modelers should favor a horizontal layout and use a more complex snake or multi-line layout whenever the diagrams are too large to fit on one page in order to support BPMN model comprehension.

show abstract

Section: Answer Timementioning

confidence: 99%

Influence of diagram layout and scrolling on understandability of BPMN processes: an eye tracking experiment with BPMN diagrams

2021

View full text Add to dashboard Cite

show abstract

“…We mitigated against this issue by filtering and reducing sources of noise, such as blink events, using temporal coarse values in time windows [40], and using multiple, redundant measures. Future work can consider alternative ways to detect stress [37] and analyze events with normalization [15,39,66].…”

Section: Limitationsmentioning

confidence: 99%

Does stress impact technical interview performance?

Behroozi

Shirolkar

Barik

et al. 2020

Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Softw

View full text Add to dashboard Cite

“…Eye gaze data is also studied with respect to certain areas of interest (AOIs) in a stimulus. AOIs are manually defined by the experimenter based on research questions and variables [10,35]. In this experiment, we consider the following AOIs: 1) bug report text, 2) entire class file containing the patched code, 3) specific methods containing the patched code, 4) entire unit test file containing tests that evaluate the patch, and 5) specific unit tests methods that evaluate the patch.…”

Section: Experiments Measuresmentioning

confidence: 99%

“…Yet, as evaluating the impact of expertise is not the goal of this study, using students as participants may be acceptable [18]. Finally, to account for conclusion validity, we choose well-documented eye-tracking metrics and analyses [34,35].…”

Section: Threats To Validitymentioning

confidence: 99%

“…These studies suggest a correlation between users' trust and their visual scanning behavior. By providing a dynamic pattern of visual attentions [16,35], eye tracking offers numerous clues to underlying cognitive processes of the participants, helps to measure workload while performing tasks [29,35,36], and determines how and when participants choose and encode information [10,29,34].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Trustworthiness Perceptions in Code Review

Bertram

Hong

Huang

et al. 2020

Proceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)

Self Cite

View full text Add to dashboard Cite

Background: Automated program repair and other bug-fixing approaches are gaining attention in the software engineering community. Automation shows promise in reducing bug fixing costs. However, many developers express reluctance about accepting machinegenerated patches into their codebases. Aims: To contribute to the scientific understanding and the empirical investigation of human trust and perception with regards to automation in software maintenance. Method: We design and conduct an eye-tracking study investigating how developers perceive trust as a function of code provenance (i.e., author or source). We systematically vary provenance while controlling for patch quality. Results: In our study of ten participants, overall visual code scanning and the distribution of attention differed across identical code patches labeled as human-vs. machine-written. Participants looked more at the source code for human-labeled patches and looked more at tests for machine-labeled patches. Participants judged human-labeled patches to have better readability and coding style. However, participants were more comfortable giving a critical task to an automated program repair tool. Conclusion: We find that there are significant differences in code review behavior based on trust as a function of patch provenance. Further, we find that important differences can be revealed by eye tracking. Our results may inform the subsequent design and analysis of automated repair techniques to increase developers' trust and, consequently, their deployment. CCS CONCEPTS • Software and its engineering → Software creation and management.

show abstract

A practical guide on conducting eye tracking studies in software engineering

Cited by 66 publications

References 95 publications

Influence of diagram layout and scrolling on understandability of BPMN processes: an eye tracking experiment with BPMN diagrams

Influence of diagram layout and scrolling on understandability of BPMN processes: an eye tracking experiment with BPMN diagrams

Does stress impact technical interview performance?

Trustworthiness Perceptions in Code Review

Contact Info

Product

Resources

About