Continuous monitoring of cost-to-go for flexible reaching control and online decisions

Abstract

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time monitoring of the expected cost-to-go, which considers target- and movement-related costs, in a novel theoretical framework. This model reproduces reaching behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous monitoring of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Embodied decision biases: individually stable across different tasks?

“…That is, decisions not only influence subsequent actions in a hierarchical, top-down fashion (Cooper and Shallice 2000 ; Newell and Simon 1972 ), but actions also bias decision-making. In support of action-based models, a number of recent studies provided empirical evidence for this bidirectional relationship for various types of actions such as manual movements like reaching (Bakker et al 2017 ; Cos et al 2021 ; de Comité, Crevecoeur and Lefèvre 2022a , b ; Michalski et al 2020 ; Nashed et al 2014 ; Pierrieau et al 2021 ) or mouse tracking (Raßbach et al 2021 ), and walking (Grießbach et al 2021 , 2022 ). Results from the latter three recent studies suggest that for both tasks, that is, walking and manual movements, the magnitude of the embodied decision biases strongly varies between participants (Grießbach et al 2021 , 2022 ; Raßbach et al 2021 ).…”

“…Action-based models argue that action information feeds back into the decision process (Lepora and Pezzulo 2015 ). Hence, it is possible to track action-related variables such as motor costs and weigh decision-making in real-time (de Comité, Lefèvre, and Crevecoeur 2022a ; b ; Wispinski et al 2020 ), an assumption that has been backed up by various experimental studies (Brenner and Smeets 2015 ; Cos et al 2021 ; de Comité, Crevecoeur, and Lefèvre 2022a , b ; Marti-Marca et al 2020 ). In the walking task, we focused on the embodied decision bias based on changes in motor costs (Grießbach et al 2021 , 2022 ).…”

A control policy can be adapted to task demands during both motor execution and motor planning

“…A modelling effort on understanding movement preparation suggests that this can be the case (Kao et al 2021). In addition, previous studies have demonstrated that it is possible to adapt the tuning of the feedback gains during ongoing movement (De Comite et al 2021, 2022, 2023), yet is currently unclear how much time this takes. De Comite and colleagues (2021) showed that the tuning of the feedback gains representing the adaptation of the control policy to the task demands was completed within 150ms.…”

“…When reaching to a narrow target, both optimal control theory and the uncontrolled manifold hypothesis would predict that any perturbation that affects hand trajectory and impedes task success should be strongly resisted. In contrast, if the target is wide and the perturbation does not reduce task success, the response to the perturbation should be much smaller as found in numerous studies (De Comite et al 2021, 2022, 2023; Nashed et al 2012; Orban de Xivry 2013; Orban de Xivry and Lefèvre 2016). The minimum intervention principle extend to error correction mechanisms as well as error are only corrected in task-relevant dimensions (van Beers et al 2013; Orban de Xivry 2013; Orban de Xivry and Lefèvre 2016).…”

Biomechanical Costs Influence Decisions Made During Ongoing Actions

“…Indeed, some models argue that sensory information about the costs and benefits of an action and its dynamics influences the decision process even after movement begins (14), making it possible to adjust an initial action during ongoing movement, as a consequence of changing environments, sudden perturbations or motor costs (14, 15, 17, 19-24). Hence, the nervous system may track action-related variables such as motor cost and weigh decision-making in real time (25), a proposal that has been supported by various experimental studies (22, 24, 26, 27).…”