Temporal binding as multisensory integration: Manipulating perceptual certainty of actions and their effects

Abstract

It has been proposed that statistical integration of multisensory cues may be a suitable framework to explain temporal binding, that is, the finding that causally related events such as an action and its effect are perceived to be shifted towards each other in time. A multisensory approach to temporal binding construes actions and effects as individual sensory signals, which are each perceived with a specific temporal precision. When they are integrated into one multimodal event, like an action-effect chain, the extent to which they affect this event’s perception depends on their relative reliability. We test whether this assumption holds true in a temporal binding task by manipulating certainty of actions and effects. Two experiments suggest that a relatively uncertain sensory signal in such action-effect sequences is shifted more towards its counterpart than a relatively certain one. This was especially pronounced for temporal binding of the action towards its effect but could also be shown for effect binding. Other conceptual approaches to temporal binding cannot easily explain these results, and the study therefore adds to the growing body of evidence endorsing a multisensory approach to temporal binding.

On the Role of Interoception in Body and Object Perception: A Multisensory-Integration Account

“…In fact, the perceived temporal attraction between the tactile event (key press) and auditory event (tone) is subject to exactly the same factors as the perceived spatial attraction between proprioceptive events (hand movement) and visual events (cursor movement). Specifically, the magnitude of both biases depends on the relative reliability of information related to the action and its effect (Cao et al, 2020; Klaffehn et al, 2021; Wolpe et al, 2013; Yamamoto, 2020) rather than on the presence of action intention (Buehner, 2012; Buehner & Humphreys, 2009; Kirsch et al, 2019; Ruess et al, 2020; Suzuki et al, 2019). Thus, action-binding and effect-binding phenomena correspond to biases toward exteroception (“bias toward audition,” i.e., an analog of bias toward vision) and interoception (bias toward body), respectively, that emerge when integrating correlated and discrepant multimodal signals.…”

Distracted to a fault: Attention, actions, and time perception

“…While our results demonstrate that such an alternative explanation seems viable, at this point we cannot make clear inferences about the mechanisms underlying these processes. However, there are at least two clear potential candidates for mediating the effect of attention on temporal binding: partial multisensory integration (e.g., Debats et al, 2017;Klaffehn et al, 2021) and heuristic judgments (Reddy, 2021).…”

“…The association of temporal binding and sense of agency is also not as clear as previously assumed, on an experimental as well as on a conceptual level (Majchrowicz & Wierzchoń, 2018;Schwarz, Weller, Klaffehn, & Pfister, 2019a). Studies have found little evidence for correlations between sense of agency measures and temporal binding (Antusch et al, 2021;Saito et al, 2015;Schwarz, Weller, Klaffehn, & Pfister, 2019a; but see Imaizumi & Tanno, 2019), and have sought to find underlying mechanisms for temporal binding spanning from causality perception to multisensory integration that are indeed not synonymous with sense of agency (Antusch et al, 2020;Hoerl et al, 2020;Klaffehn et al, 2021). Moreover, the measuring of temporal binding has come under scrutiny with neither different measures of temporal binding nor subcomponents of the same temporal binding measure relating to one another (Siebertz & Jansen, 2022;Tonn et al, 2021).…”

“…1). This distraction of attentional resources, in turn, could facilitate partial multisensory integration (e.g., Debats et al, 2017;Klaffehn et al, 2021) leading to a higher perceptual overlap of both events. For example, it is quite possible that focus of attentional resources may change perceptual certainty of either event, thus affecting multisensory integration.…”

Sense of agency in social hierarchies.

“…Finally, we deliberately chose to focus on explicit sense of agency ratings in this series of experiments in the experimental procedure as well as for data interpretation; for one, because we wanted to analyze different rating scales that are either used synonymously in experiments or associated in theory, and for another, because implicit measures of sense of agency are currently very controversially discussed in their association with sense of agency (e.g., Hoerl et al, 2020; Klaffehn et al, 2021; Schwarz, Pfister, et al, 2018, 2019; Schwarz & Weller, 2022; Siebertz & Jansen, 2022; Tonn et al, 2021). Nevertheless, explicit ratings automatically invite potential biases.…”