Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding

Lins, Jonas; Schöner, Gregor

doi:10.3758/s13414-019-01847-9

Cited by 10 publications

(7 citation statements)

References 105 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our model, the direct comparison between the accuracy and reaction times (measured in the overall task and critical blocks) collected in the RT- vs the MT-race-IAT task revealed a significantly higher accuracy and longer RTs in the MT-race-IAT (see Table 2 ). This evidence is consistent with recent findings [ 15 , 22 , 25 , 26 , 64 ] and emphasises the impact of the strategic motor corrections engaged during the temporal unfolding of the hand movements to reach the selected category [ 17 – 20 ].…”

Section: Discussionsupporting

confidence: 93%

The biased hand. Mouse-tracking metrics to examine the conflict processing in a race-implicit association test

et al. 2022

View full text Add to dashboard Cite

In this study, we adapted a race-Implicit Association Test (race-IAT) to mouse-tracking (MT) technique to identify the more representative target observed MT-metrics and explore the temporal unfolding of the cognitive conflict emerging during the categorisation task. Participants of Western European descent performed a standard keyboard-response race-IAT (RT-race-IAT) and an MT-race-IAT with the same structure. From a behavioural point of view, our sample showed a typical Congruency Effect, thus a pro-White implicit bias, in the RT-race-IAT. In addition, in the MT-race-IAT, the MT-metrics showed a similar Congruency Effect mirroring the higher attraction of the averaged-trajectories towards the incorrect response button in incongruent than congruent trials. Moreover, these MT-metrics were positively associated with RT-race-IAT scores, strengthening the MT approach’s validity in characterising the implicit bias. Furthermore, the distributional analyses showed that mouse trajectories displayed a smooth profile both in congruent and incongruent trials to indicate that the unfolding of the decision process and the raised conflict is guided by dynamical cognitive processing. This latter continuous competition process was studied using a novel phase-based approach which allowed to temporally dissect an Early, a Mid and a Late phase, each of which may differently reflect the decision conflict between automatic and controlled responses in the evolution of the mouse movement towards the target response. Our results show that the MT approach provides an accurate and finer-grained characterisation of the implicit racial attitude than classical RT-IAT. Finally, our novel phase-based approach can be an effective tool to shed light on the implicit conflict processing emerging in a categorisation task with a promising transferable value in different cognitive and neuropsychological fields.

show abstract

Section: Discussionsupporting

confidence: 93%

The biased hand. Mouse-tracking metrics to examine the conflict processing in a race-implicit association test

et al. 2022

View full text Add to dashboard Cite

show abstract

“…That is, with mouse-tracking it is possible to quantify the stages by which the decision process unfolds (e.g., Kieslich, Henninger, Wulff, Haslbeck, & Schulte-Mecklenbeck, 2019). Accordingly, mousetracking has been successfully used across a large number of domains such as language (Lins, & Schöner, 2019;Spivey, Grosjean, & Knoblich, 2005), social cognition (Freeman, Dale, & Farmer, 2011;Freeman, Pauker, & Sanchez, 2016) and memory (Papesh, & Goldinger, 2012;Papesh, Hicks, & Guevara Pinto, 2019).…”

Section: Introductionmentioning

confidence: 99%

Predicting hand movements with distributional semantics: evidence from mouse-tracking

Gatti¹,

Marelli²,

Rinaldi³

2021

Preprint

View full text Add to dashboard Cite

Despite mouse-tracking has been taken as a real-time window on different aspects of human decision-making processes, whether purely semantic information affects response conflict at the level of motor output is still unknown. Here, we investigated the possible effects of semantic knowledge on hand movements by predicting participants’ performance in a mouse-tracking task through distributional semantics, a usage-based modelling approach to meaning. Participants were shown word pairs and were required to perform a two-alternative forced choice task by moving the computer mouse and selecting either the more abstract or the more concrete word, depending on the experimental condition. Results showed that mouse trajectories were affected by the semantic relatedness of the word-pairs as indexed by the distributional semantic model, despite this dimension was not relevant for the task at hand. In particular, mouse trajectories reflected the response conflict and its temporal evolution, with larger deviation and movement uncertainty for increasing word semantic relatedness. These findings testify the effects of word meaning on decision-making processes as indexed by a mouse-tracking conflict. They also document a strict interplay between sensory-motor processes and the structure of human semantic memory.

show abstract

“…Such extensions may be useful if motor behavior could provide evidence for the postulated neural processes of perceptual grounding. We have made a first step toward such evidence (Lins & Schöner, 2019). In a variant of the mouse tracking paradigm (Spivey & Dale, 2006), participants moved the mouse‐controlled cursor onto the target designated by a relational phrase.…”

Section: Discussionmentioning

confidence: 99%

A Neural Dynamic Model of the Perceptual Grounding of Spatial and Movement Relations

2021

Self Cite

View full text Add to dashboard Cite

How does the human brain link relational concepts to perceptual experience? For example, a speaker may say "the cup to the left of the computer" to direct the listener's attention to one of two cups on a desk. We provide a neural dynamic account for both perceptual grounding, in which relational concepts enable the attentional selection of objects in the visual array, and for the generation of descriptions of the visual array using relational concepts. In the model, activation in neural populations evolves dynamically under the influence of both inputs and strong interaction as formalized in dynamic field theory. Relational concepts are modeled as patterns of connectivity to perceptual representations. These generalize across the visual array through active coordinate transforms that center the representation of target objects in potential reference objects. How the model perceptually grounds or generates relational descriptions is probed in 104 simulations that systematically vary the spatial and movement relations employed, the number of feature dimensions used, and the number of matching and nonmatching objects. We explain how sequences of decisions emerge from the time-and state-continuous neural dynamics, how relational hypotheses are generated and either accepted or rejected, followed by the selection of new objects or the generation of new relational hypotheses. Its neural realism distinguishes the model from information processing accounts, its capacity to autonomously generate sequences of processing steps distinguishes it from deep neural network accounts. The model points toward a neural dynamic theory of higher cognition.

show abstract

Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding

Cited by 10 publications

References 105 publications

The biased hand. Mouse-tracking metrics to examine the conflict processing in a race-implicit association test

The biased hand. Mouse-tracking metrics to examine the conflict processing in a race-implicit association test

Predicting hand movements with distributional semantics: evidence from mouse-tracking

A Neural Dynamic Model of the Perceptual Grounding of Spatial and Movement Relations

Contact Info

Product

Resources

About