Eyes are different than Hands

Jungwirth, Florian; Murauer, Michaela; Haslgrübler, Michael; Ferscha, Alois

doi:10.1145/3197768.3201565

Cited by 8 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are three types of interesting eye movements when observing visual attention: fixation, saccades, and smooth pursuits (Duchowski, 2017 ). ET has, for example, been used in an industrial context with gaze as machine control input (Jungwirth et al, 2018 ), to evaluate new ways to facilitate human–robot communication (Tang et al, 2019 ), analyze the navigational intent in humans and how they interact with autonomous forklifts (Chadalavada et al, 2020 ), and investigate pedestrians' understanding of an autonomous vehicle's intention to stop at a simulated road crossing (Hochman et al, 2020 ).…”

Section: Introductionmentioning

confidence: 99%

Comparison of LSTM, Transformers, and MLP-mixer neural networks for gaze based human intention prediction

Pettersson

Falkman

2023

Front. Neurorobot.

View full text Add to dashboard Cite

Collaborative robots have gained popularity in industries, providing flexibility and increased productivity for complex tasks. However, their ability to interact with humans and adapt to their behavior is still limited. Prediction of human movement intentions is one way to improve the robots adaptation. This paper investigates the performance of using Transformers and MLP-Mixer based neural networks to predict the intended human arm movement direction, based on gaze data obtained in a virtual reality environment, and compares the results to using an LSTM network. The comparison will evaluate the networks based on accuracy on several metrics, time ahead of movement completion, and execution time. It is shown in the paper that there exists several network configurations and architectures that achieve comparable accuracy scores. The best performing Transformers encoder presented in this paper achieved an accuracy of 82.74%, for predictions with high certainty, on continuous data and correctly classifies 80.06% of the movements at least once. The movements are, in 99% of the cases, correctly predicted the first time, before the hand reaches the target and more than 19% ahead of movement completion in 75% of the cases. The results shows that there are multiple ways to utilize neural networks to perform gaze based arm movement intention prediction and it is a promising step toward enabling efficient human-robot collaboration.

show abstract

Section: Introductionmentioning

confidence: 99%

Comparison of LSTM, Transformers, and MLP-mixer neural networks for gaze based human intention prediction

Pettersson

Falkman

2023

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

Eye-tracking support for analyzing human factors in human-robot collaboration during repetitive long-duration assembly processes

Gervasi,

Capponi,

Mastrogiacomo

et al. 2024

Prod. Eng. Res. Devel.

View full text Add to dashboard Cite

The integration of eye-tracking technology in manufacturing is emerging as a powerful tool for optimizing human performance and well-being in the workplace. Advances in various fields enabled the recent development of smaller, wearable, and wireless eye-tracking devices which are suitable for naturalistically studying manufacturing processes, such as human-robot collaboration (HRC). However, the implementation of eye-tracking for evaluating mental workload in HRC is still limited, especially in long-duration sessions. This paper provides an overview on the application of eye-tracking technology in the context of cognitive ergonomics within the manufacturing sector, with special attention to eye-tracking metrics and their interpretation relatively to human state in long-duration sessions (i.e., work shifts). In addition, an example case study will be presented to explore the reliability of the most common eye-tracking metrics, concerning a repetitive assembly process of 8 h in an HRC setting. Among the explored eye-tracking metrics, pupil dilation, number and average duration of fixations, and number saccades provided useful insights on the mental strain in dynamic conditions. In addition, from the multiple information gathered by eye-tracking, different phenomena related to mental workload were able to be distinguished. The use of cognitive resources resulting from learning process was well detected by pupil dilation, number of fixations and saccades. Mental fatigue, on the other hand, was well detected by the average duration of fixations and the pupil diameter. These results highlight the need to consider multiple eye-tracking metrics simultaneously to obtain a holistic view of the operator’s psychophysiological state.

show abstract

Human Movement Direction Classification using Virtual Reality and Eye Tracking

Pettersson

Falkman

2020

Procedia Manufacturing

View full text Add to dashboard Cite

Eyes are different than Hands

Cited by 8 publications

References 20 publications

Comparison of LSTM, Transformers, and MLP-mixer neural networks for gaze based human intention prediction

Comparison of LSTM, Transformers, and MLP-mixer neural networks for gaze based human intention prediction

Eye-tracking support for analyzing human factors in human-robot collaboration during repetitive long-duration assembly processes

Human Movement Direction Classification using Virtual Reality and Eye Tracking

Contact Info

Product

Resources

About