Acquiring reusable skills in intrinsically motivated reinforcement learning

Farahani, Marzieh Davoodabadi; Mozayani, Nasser

doi:10.1007/s10845-020-01629-3

Cited by 5 publications

(3 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another direction in HRL is proposing methods based on intrinsic motivations for skill acquisition. Farahani et al [36] propose an incremental model for acquiring and evaluating task-independent skills in intrinsically motivated RL. The learning process contains two phases.…”

Section: Related Workmentioning

confidence: 99%

Speeding up hierarchical reinforcement learning using state-independent temporal skills

Azadkhah,

Davoodi,

Ghazanfari

et al. 2024

Preprint

View full text Add to dashboard Cite

Hierarchical reinforcement learning has the potential to expedite long-term decision-making by abstracting policies into multiple levels. Encouraging outcomes have been observed in challenging reward environments through the use of skills, defined as sequences of basic actions. While existing methods based on offline data have shown promise, the resulting lower-level policies may suffer from unreliability due to limited demonstration coverage or shifts in distribution. To address this limitation, we propose a novel approach to autonomously identify state-independent temporal skills (SITS) by extracting the most repetitive action sequences from a trained agent. These skills, acquired in a simpler source task, can then be transferred to a more complex target task to enhance the agent’s training efficiency, particularly during the exploration phase. Our method is independent of other hierarchical reinforcement learning techniques and can be used in conjunction with them. Experimental results demonstrate the efficacy of incorporating SITS in addressing complex RL challenges.

show abstract

Section: Related Workmentioning

confidence: 99%

Speeding up hierarchical reinforcement learning using state-independent temporal skills

Azadkhah,

Davoodi,

Ghazanfari

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Although, this method is poorly scalable since it does not have a mechanism for an automatic sub-goal extraction. In Davoodabadi Farahani and Mozayani [ 19 ], authors utilize different IM heuristics for both goal discovery and exploration with the Options Framework under-hood. They divide the learning process into two separate stages to overcome intrinsic and extrinsic reward interference, consequently, requiring an explicit indication of a goal change.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical intrinsically motivated agent planning behavior with dreaming in grid environments

et al. 2022

View full text Add to dashboard Cite

Biologically plausible models of learning may provide a crucial insight for building autonomous intelligent agents capable of performing a wide range of tasks. In this work, we propose a hierarchical model of an agent operating in an unfamiliar environment driven by a reinforcement signal. We use temporal memory to learn sparse distributed representation of state–actions and the basal ganglia model to learn effective action policy on different levels of abstraction. The learned model of the environment is utilized to generate an intrinsic motivation signal, which drives the agent in the absence of the extrinsic signal, and through acting in imagination, which we call dreaming. We demonstrate that the proposed architecture enables an agent to effectively reach goals in grid environments.

show abstract

“…21,22 The skills acquired by the computational agents can be evaluated by using the intrinsic motivations in the RL framework. 23 This optimal policy learned by the computational agent can be transferred to the robots for its optimal locomotion. 24,25 The authors have utilized two different algorithms, viz, extended single-agent Q-learning algorithm and Team Q-learning algorithm, for a fully cooperative multi-robot box pushing task.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning based path planning of multiple agents of SwarmItFIX robot for fixturing operation in sheetmetal milling process

Veeramani

Muthuswamy

2022

Proceedings of the Institution of Mechanical Engineers, Part B:

View full text Add to dashboard Cite

SwarmItFIX (self-reconfigurable intelligent swarm fixtures) is a multi-agent setup mainly used as a robotic fixture for large Sheet metal machining operations. A Constraint Satisfaction Problem (CSP) based planning model is utilized currently for computing the locomotion sequence of multiple agents of the SwarmItFIX. But the SwarmItFIX faces several challenges with the current planner as it fails on several occasions. Moreover, the current planner computes only the goal positions of the base agent, not the path. To overcome these issues, a novel hierarchical planner is proposed, which employs Monte Carlo and SARSA TD based model-free Reinforcement Learning (RL) algorithms for the computation of locomotion sequences of head and base agents, respectively. These methods hold two distinct features when compared with the existing methods (i) the transition model is not required for obtaining the locomotion sequence of the computational agent, and (ii) the state-space of the computational agent become scalable. The obtained results show that the proposed planner is capable of delivering optimal makespan for effective fixturing during the sheet metal milling process.

show abstract

Acquiring reusable skills in intrinsically motivated reinforcement learning

Cited by 5 publications

References 28 publications

Speeding up hierarchical reinforcement learning using state-independent temporal skills

Speeding up hierarchical reinforcement learning using state-independent temporal skills

Hierarchical intrinsically motivated agent planning behavior with dreaming in grid environments

Reinforcement learning based path planning of multiple agents of SwarmItFIX robot for fixturing operation in sheetmetal milling process

Contact Info

Product

Resources

About