Data-driven Simulation Framework for Expressive Piano Playing by Anthropomorphic Hand with Variable Passive Properties

Wang, Huijiang; Howison, Toby; Hughes, Josie; Abdulali, Arsen; Iida, Fumiya

doi:10.1109/robosoft54090.2022.9762138

Cited by 9 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The combination of both simulation and physical experimentation give participants insights into real-world problems, and find the potential solutions to these problems. The complementary experiments would also serve as an educational benchmark tutorial for participants to learn how skills and optimizations implemented in simulation can be transferred to a real hardware set with minimized reality gap (i.e., sim2real transfer [14]).…”

Section: B Survey Studymentioning

confidence: 99%

Machine Learning for Soft Robot Sensing and Control: A Tutorial Study

Wang

Thuruthel

Gilday

et al. 2022

2022 IEEE 5th International Conference on Industrial Cyber-Physical Systems (ICPS)

Self Cite

View full text Add to dashboard Cite

Developing feedback controllers for robots with embedded sensors is challenging and typically requires expert knowledge. As machine learning (ML) advances, the development of learning-based controllers has become more and more accessible, even to non-experts. This work presents the development of a tutorial to educate non-roboticists about MLbased sensing and control in cyber-physical systems using a soft robotic device. We demonstrated this by creating a recurrent neural network-based closed-loop force controller for a soft finger with embedded soft sensors. Our hypothesis is validated in a 2.5hour workshop session for students with no prior knowledge of robot control. This work serves as a tutorial for participants aiming to experience and perform a general benchmark for soft robot control tasks, with little or even no expertise in robotics.

show abstract

Section: B Survey Studymentioning

confidence: 99%

Machine Learning for Soft Robot Sensing and Control: A Tutorial Study

Wang

Thuruthel

Gilday

et al. 2022

2022 IEEE 5th International Conference on Industrial Cyber-Physical Systems (ICPS)

Self Cite

View full text Add to dashboard Cite

show abstract

“…A neuromusculoskeletal model in silico with a bridged simto-real gap is promising to address this challenge. Piano playing is a challenge that is particularly interesting for humans as it requires extreme dexterity, adaptability, and behavioral richness to achieve a range of expressive playing styles [49]. A number of researchers have aimed to build both physical prototypes [50][51][52][53][54][55][56][57], data-driven virtual prototypes [49,58] and non-data-driven simulation approach [59] to manipulate this complex musical instrument.…”

Section: Introductionmentioning

confidence: 99%

“…Piano playing is a challenge that is particularly interesting for humans as it requires extreme dexterity, adaptability, and behavioral richness to achieve a range of expressive playing styles [49]. A number of researchers have aimed to build both physical prototypes [50][51][52][53][54][55][56][57], data-driven virtual prototypes [49,58] and non-data-driven simulation approach [59] to manipulate this complex musical instrument. However, they have failed to mimic the biological neural and muscular activities as well as the complex passive dynamics between the interaction of a physiological-accurate body and a piano, resulting in a significant reality gap between the model and physics.…”

Section: Introductionmentioning

confidence: 99%

Coordinating upper limbs for octave playing on the piano via neuro-musculoskeletal modeling

Wang,

Nonaka,

Abdulali

et al. 2023

Bioinspir. Biomim.

View full text Add to dashboard Cite

Understanding the coordination of multiple biomechanical degrees of freedom in biological organisms is crucial for unraveling the neurophysiological control of sophisticated motor tasks. This study focuses on the cooperative behavior of upper-limb motor movements in the context of octave playing on the piano. While the vertebrate locomotor system has been extensively investigated, the coherence and precision timing of rhythmic movements in the upper-limb system remain incompletely understood. Inspired by the spinal cord neuronal circuits (central pattern generator, CPG), a computational neuro-musculoskeletal model is proposed to explore the coordination of upper-limb motor movements during octave playing across varying tempos and volumes. The proposed model incorporates a CPG-based nervous system, a physiologically-informed mechanical body, and a piano environment to mimic human joint coordination and expressiveness. The model integrates neural rhythm generation, spinal reflex circuits, and biomechanical muscle dynamics while considering piano playing quality and energy expenditure. Based on real-world human subject experiments, the model has been refined to study tempo transitions and volume control during piano playing. This computational approach offers insights into the neurophysiological basis of upper-limb motor coordination in piano playing and its relation to expressive features.

show abstract

“…[11] Furthermore, soft hands open up additional functionalities that can be exploited for versatile manipulation strategies like the use of environmental constraints [5] and programmable anisotropic stiffness properties. [7,12] When it comes to dynamic manipulation tasks, soft robotic designs have shown surprisingly robust behaviors. The RBO hand 3 [13,14] displays robust manipulation capabilities when reorienting variable objects with a manually trained sequence of finger moves.…”

Section: Introductionmentioning

confidence: 99%

Predictive Learning of Error Recovery with a Sensorized Passivity‐Based Soft Anthropomorphic Hand

Gilday

Thuruthel

Iida

2023

Advanced Intelligent Systems

Self Cite

View full text Add to dashboard Cite

Manipulation strategies based on the passive dynamics of soft-bodied interactions provide robust performances with limited sensory information. They utilize the kinematic structure and passive dynamics of the body to adapt to objects of varying shapes and properties. However, these soft passive interactions make the state of the robotic device influenced by the environment, making control generation and state estimation difficult. This work presents a closed-loop framework for dynamic interaction-based grasping that relies on two novelties: 1) a wrist-driven passive soft anthropomorphic hand that can generate robust grasp strategies using one-step kinaesthetic teaching and 2) a learning-based perception system that uses temporal data from sparse tactile sensors to predict and adapt to failures before it happens. With the anthropomorphic soft design and wrist-driven control, it is shown that controllers can be generated robust to novel objects and location uncertainty. With the learning-based high-level perception system and 32 sensing receptors, it is shown that failures can be predicted in advance, further improving the robustness of the entire system by more than doubling the grasping success rate. From over 1000 real-world grasping trials, both the control and perception framework are also seen to be transferable to novel objects and conditions. An interactive preprint version of the article can be found here:

show abstract

Data-driven Simulation Framework for Expressive Piano Playing by Anthropomorphic Hand with Variable Passive Properties

Cited by 9 publications

References 14 publications

Machine Learning for Soft Robot Sensing and Control: A Tutorial Study

Machine Learning for Soft Robot Sensing and Control: A Tutorial Study

Coordinating upper limbs for octave playing on the piano via neuro-musculoskeletal modeling

Predictive Learning of Error Recovery with a Sensorized Passivity‐Based Soft Anthropomorphic Hand

Contact Info

Product

Resources

About