The World as an External Memory: The Price of Saccades in a Sensorimotor Task

Abstract

Theories of embodied cognition postulate that the world can serve as an external memory. This implies that instead of storing visual information in working memory the information may be equally retrieved by appropriate eye movements. Given this assumption, the question arises, how we balance the effort of memorization with the effort of visual sampling our environment. We analyzed eye-tracking data in a sensorimotor task where participants had to produce a copy of a LEGO®-blocks-model displayed on a computer screen. In the unconstrained condition, the model appeared immediately after eye-fixation on the model. In the constrained condition, we introduced a 0.7 s delay before uncovering the model. The model disappeared as soon as participants made a saccade outside of the Model Area. To successfully copy a model of 8 blocks participants made saccades to the Model Area on average 7.9 times in the unconstrained condition and 5.2 times in the constrained condition. However, the mean duration of a trial was 2.9 s (14%) longer in the constrained condition even when taking into account the delayed visibility of the model. In this study, we found evidence for an adaptive shift in subjects’ behavior toward memorization by introducing a price for a certain type of saccades. However, the response is not adaptive; it is maladaptive, as memorization leads to longer overall performance time.

When Natural Behavior Engages Working Memory

“…Here, memory use is weighted against locomotive effort, and depending on the reliability of the representation, the actor would rely on the information in mind or update it. 33 Looking back can be rather “cheap,” if only a few saccades are required, 34 , 35 , 36 but becomes more costly in an ecological context as we have to move the head, arms, and body. 11 , 37 , 38 Further, constrained screen-based tasks often require participants to remain still and hold gaze on a single spot.…”

When does the Brain Ask for Help from the Eyes?

“…Melnik et al [1] conducted a ground-breaking study to quantify the trade-off between the use of eye movements for working-memory or for using the outside world as a working memory (during purposeful actions). This trade-off is a basic trait of vision training with shutter glasses: the eyes become intermittently blinded, so they are forced to switch between internal memory and 'the world', i.e.…”

“…external memory. In Melnik et al study [1], was the cost for a new sample of visual information that participants had to "pay" a short visual delay. The idea of 'payment' by delay or a time penalty is also used in [35].…”

“…What happens when information to remember becomes too much for the brain? Apparently, we turn to our eyes for help to use the brain's ability to see the world as an external memory [1]. This "Embodied Cognition" postulate by Clark and Chalmers [39] says that instead of storing visual information in working memory: it may be equally retrieved by appropriate eye movements [40].…”

“…

We report a diagnostic tool to distinguish erratic from other eye movements. We explain its properties and successfully apply it to the illustrative experiment by Melnik et al [1]. The tool is based upon entropy measurement of a stochastic model, as developed in complexity theory.

…”