Reinforcement meta-learning optimizes visuomotor learning

Sugiyama, Taisei; Schweighofer, Nicolas; Izawa, Jun

doi:10.1101/2020.01.19.912048

Cited by 3 publications

(3 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A current challenge is to learn suitable action selection policies. Although explorationexploitation is known to be central in motor learning , it is yet unclear what process drives action selection in the brain (Carland et al, 2019;Sugiyama et al, 2020). These approaches still need to be experimentally assessed in a motor learning context.…”

Section: Discussionmentioning

confidence: 99%

Machine Learning Approaches for Motor Learning: A Short Review

Caramiaux

Françoise

Liu

et al. 2020

Front. Comput. Sci.

View full text Add to dashboard Cite

Machine learning approaches have seen a considerable number of applications in human movement modeling but remain limited for motor learning. Motor learning requires that motor variability be taken into account and poses new challenges because the algorithms need to be able to differentiate between new movements and variation in known ones. In this short review, we outline existing machine learning models for motor learning and their adaptation capabilities. We identify and describe three types of adaptation: Parameter adaptation in probabilistic models, Transfer and meta-learning in deep neural networks, and Planning adaptation by reinforcement learning. To conclude, we discuss challenges for applying these models in the domain of motor learning support systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Machine Learning Approaches for Motor Learning: A Short Review

Caramiaux

Françoise

Liu

et al. 2020

Front. Comput. Sci.

View full text Add to dashboard Cite

show abstract

“…Finally, a higher-level mechanism that governs the way of adaptation or learning has been proposed as meta-learning, which is mainly investigated in the framework of reinforcement learning [35,36]. An apparent different strategy to the drawing task found in two subjects in the present study might be a resultant outcome of a kind of meta-learning because the subjects had acquired the skill to perform the task through a reward-based training process.…”

Section: The Saccades and The Control Model Of An Arm Movementmentioning

confidence: 92%

Functional Roles of Saccades for a Hand Movement

2020

View full text Add to dashboard Cite

An eye saccade provides appropriate visual information for motor control. The present study was aimed to reveal the role of saccades in hand movements. Two types of movements, i.e., hitting and circle-drawing movements, were adopted, and saccades during the movements were classified as either a leading saccade (LS) or catching saccade (CS) depending on the relative gaze position of the saccade to the hand position. The ratio of types of the saccades during the movements was heavily dependent on the skillfulness of the subjects. In the late phase of the movements in a less skillful subject, CS tended to occur in less precise movements, and precision of the movement tended to be improved in the subsequent movement in the hitting. While LS directing gaze to a target point was observed in both types of the movements regardless of skillfulness of the subjects, LS in between a start point and a target point, which led gaze to a local minimum variance point on a hand movement trajectory, was exclusively found in the drawing in a less skillful subject. These results suggest that LS and some types of CS may provide positional information of via-points in addition to a target point and visual information to improve precision of a feedforward controller in the brain, respectively.

show abstract

“…In addition, it is possible to learn the reward derivation mechanism by using the inverse reinforcement learning model [36,37]. In this case, unlike the previous approach, a meta-cognitive artificial intelligence model that can adapt to other environments instead of just one environment is developed [38,39].…”

Section: Explainable Meta-reinforcement Learning (Xmrl)mentioning

confidence: 99%

Explainable Artificial Intelligence (xAI) Approaches and Deep Meta-Learning Models

Dağlarlı¹

2020

Advances and Applications in Deep Learning

View full text Add to dashboard Cite

The explainable artificial intelligence (xAI) is one of the interesting issues that has emerged recently. Many researchers are trying to deal with the subject with different dimensions and interesting results that have come out. However, we are still at the beginning of the way to understand these types of models. The forthcoming years are expected to be years in which the openness of deep learning models is discussed. In classical artificial intelligence approaches, we frequently encounter deep learning methods available today. These deep learning methods can yield highly effective results according to the data set size, data set quality, the methods used in feature extraction, the hyper parameter set used in deep learning models, the activation functions, and the optimization algorithms. However, there are important shortcomings that current deep learning models are currently inadequate. These artificial neural network-based models are black box models that generalize the data transmitted to it and learn from the data. Therefore, the relational link between input and output is not observable. This is an important open point in artificial neural networks and deep learning models. For these reasons, it is necessary to make serious efforts on the explainability and interpretability of black box models.

show abstract

Reinforcement meta-learning optimizes visuomotor learning

Cited by 3 publications

References 49 publications

Machine Learning Approaches for Motor Learning: A Short Review

Machine Learning Approaches for Motor Learning: A Short Review

Functional Roles of Saccades for a Hand Movement

Explainable Artificial Intelligence (xAI) Approaches and Deep Meta-Learning Models

Contact Info

Product

Resources

About