Brain Representations of Motion Generation and Perception: Space-Time Geometries and the Arts

Flash, Tamar

doi:10.1007/978-3-030-57227-3_1

Cited by 3 publications

(3 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In human development and artistic expression, motor behavior is intricately intertwined with perception(1; 2; 3; 4; 5). This fundamental notion is well-established in psychological and psychophysical studies.…”

Section: Mainmentioning

confidence: 99%

Play it by Ear: A perceptual algorithm for autonomous melodious piano playing with a bio-inspired robotic hand

Azadjou,

Marjaninejad,

Valero-Cuevas

2024

Preprint

View full text Add to dashboard Cite

Perception shapes the learning and performance of motor behavior in animals. In contrast to this inherent biological and psychological connection between perception and action, traditional artificial intelligence methods for robotics emphasize reward-driven extensive trial-and-error or error-driven control techniques.Our study goes back to the perceptual roots of biological learning and behavior, and demonstrates a novel end-to-end perceptual experience-driven approach for autonomous piano playing. Our ‘Play it by Ear’ perceptual learning algorithm, coupled to a bio-inspired 4-finger robotic hand, can replicate melodies on a keyboard after hearing them once—without explicit or prior knowledge of notes, the hand, or the keyboard. Our key innovation is an end-to-end pipeline that, after a brief period of ‘motor babbling’ by the hand, converts the sound of a melody into native musical percepts (note sequences and intensities) that it replays as sequences of key presses.In this way, any new melody consisting of notes experienced during babbling can be reproduced by the robotic musician hand on the basis of its percepts. This playback includes capturing the qualitative and quantitative musical dynamics and tempo with a nuance comparable with that of four human pianists performing the same melody. These compelling results emphasize the perceptual underpinnings of artistic performance as an alternative to traditional control-theoretical emphasis on state estimation and error correction. Our approach opens avenues for the development of simple machines that can still execute artistic and physical tasks that approach the nuance inherent in human behavior.

show abstract

Section: Mainmentioning

confidence: 99%

Play it by Ear: A perceptual algorithm for autonomous melodious piano playing with a bio-inspired robotic hand

Azadjou,

Marjaninejad,

Valero-Cuevas

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…are coupled in a specific way. These primitives are specifically parameterized or optimized to fit motor needs for specific tasks, such as locomotion (Ivanenko et al, 2004;Merkle et al, 1998) or arm movements (d'Avella et al, 2006;d'Avella and Lacquaniti, 2013;Flash and Berthoz, 2021). Though synergies are somewhat debated, there is evidence for muscle synergies (Bizzi et al, 1995;Tresch et al, 1999;Mussa-Ivaldi et al, 1994;Ivanenko et al, 2004).…”

Section: Compositionalitymentioning

confidence: 99%

Human Arm Redundancy - A New Approach for the Inverse Kinematics Problem

Barliya

Krausz

Naaman

et al. 2023

Preprint

View full text Add to dashboard Cite

The inverse kinematics problem deals with the question of how the nervous system coordinates movement to resolve redundancy, such as in the case of arm reaching movements where more degrees of freedom are available at the joint versus hand level. In particular, this work focuses on determining which coordinate frames can best represent human movements, allowing the motor system to solve the inverse kinematics problem in the presence of kinematic redundancies. In particular, in this work we used a multi-dimension sparse source separation method called FADA to derive sets of basis functions (here called sources) for both the task and joint spaces, with joint space being represented in terms of either the absolute or anatomical joint angles. We assessed the similarities between the joint and task sources in each of these joint representations. We found that the time-dependent profiles of the absolute reference frame's sources show greater similarity to those of the corresponding sources in the task space. This result was found to be statistically significant. Hence, our analysis suggests that the nervous system represents multi-joint arm movements using a limited number of basis functions, to allow for simple transformations between task and joint spaces. Importantly, joint space seems to be represented in terms of an absolute reference frame to achieve successful performance and simplify inverse kinematics transformations in the face of the existing kinematic redundancies. Further studies will be needed to determine the generalizability of this finding and its implications concerning neural control of movement.

show abstract

“…Neuropsychological assessments (NPAs) are often used to identify cognitive impairments (CIs) [6]. Researchers from a variety of fields, including mathematics [7][8][9][10], robotics [11][12][13][14], neuropsychology [15], and computer science [16], are interested in researching visuospatial memory locomotion and NPAs associated with it to develop a stereotypical model of human navigational behavior.…”

Section: Introductionmentioning

confidence: 99%

Clustering analysis of human navigation trajectories in a visuospatial memory locomotor task using K-Means and hierarchical agglomerative clustering

Annaki¹,

Rahmoune²,

Bourhaleb³

et al. 2022

E3S Web Conf.

View full text Add to dashboard Cite

Throughout this study, we employed unsupervised machine learning clustering algorithms, namely K-Means [1] and hierarchical agglomerative clustering (HAC) [2], to explore human locomotion and wayfinding using a VR Magic Carpet (VMC) [3], a table test version known as the Corsi Block Tapping task (CBT) [4]. This variation was carried out in the context of a virtual reality experimental setup. The participants were required to memorize a sequence of target positions projected on the rug and walk to each target figuring in the displayed sequence. the participant’s trajectory was collected and analyzed from a kinematic perspective. An earlier study [5] identified three different categories, but the classification remained ambiguous, implying that they include both kinds of individuals (normal and patients with cognitive spatial impairments). On this basis, we utilized K-Means and HAC to distinguish the navigation behavior of patients from normal individuals, emphasizing the most important discrepancies and then delving deeper to gain more insights.

show abstract

Brain Representations of Motion Generation and Perception: Space-Time Geometries and the Arts

Cited by 3 publications

References 74 publications

Play it by Ear: A perceptual algorithm for autonomous melodious piano playing with a bio-inspired robotic hand

Play it by Ear: A perceptual algorithm for autonomous melodious piano playing with a bio-inspired robotic hand

Human Arm Redundancy - A New Approach for the Inverse Kinematics Problem

Clustering analysis of human navigation trajectories in a visuospatial memory locomotor task using K-Means and hierarchical agglomerative clustering

Contact Info

Product

Resources

About