Multi-Task Hierarchical Imitation Learning for Home Automation

Fox, Roy; Berenstein, Ron; Stoica, Ion; Goldberg, Ken

doi:10.1109/coase.2019.8843293

Cited by 25 publications

(26 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A number of approaches have explored the learning of primitives or options from demonstrations together with a high-level controller that is either learned from demonstrations [Kroemer et al, 2015, Krishnan et al, 2017, Ding et al, 2019, Lynch et al, 2020, or learned from interactions with the environment [Manschitz et al, 2015, Kipf et al, 2019, Shankar et al, 2019, or hand specified [Pastor et al, 2009, Fox et al, 2019. AQuaDem can be loosely interpreted as a two-level procedure as well, where the primitives (action discretization step) are learned fully offline, however there is no concept of goal nor temporally extended actions.…”

Section: Hierarchical Imitation Learningmentioning

confidence: 99%

Continuous Control with Action Quantization from Demonstrations

Dadashi¹,

Hussenot²,

Vincent³

et al. 2021

Preprint

View full text Add to dashboard Cite

In Reinforcement Learning (RL), discrete actions, as opposed to continuous actions, result in less complex exploration problems and the immediate computation of the maximum of the action-value function which is central to dynamic programming-based methods. In this paper, we propose a novel method: Action Quantization from Demonstrations (AQuaDem) to learn a discretization of continuous action spaces by leveraging the priors of demonstrations. This dramatically reduces the exploration problem, since the actions faced by the agent not only are in a finite number but also are plausible in light of the demonstrator's behavior. By discretizing the action space we can apply any discrete action deep RL algorithm to the continuous control problem. We evaluate the proposed method on three different setups: RL with demonstrations, RL with play data -demonstrations of a human playing in an environment but not solving any specific task-and Imitation Learning. For all three setups, we only consider human data, which is more challenging than synthetic data. We found that AQuaDem consistently outperforms state-of-the-art continuous control methods, both in terms of performance and sample efficiency. We provide visualizations and videos in the paper's website: https://google-research.github.io/aquadem/.

show abstract

Section: Hierarchical Imitation Learningmentioning

confidence: 99%

Continuous Control with Action Quantization from Demonstrations

Dadashi¹,

Hussenot²,

Vincent³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We conducted a limited number of experiments-4 sample scenes per 7, 9 and 11 objects. We then ran both the HLP and the ST O producing a total of 24 robot runs 7 . Sample results are shown in Fig.…”

Section: ) Real Robot Experimentsmentioning

confidence: 99%

Human-like Planning for Reaching in Cluttered Environments

Hasan

Warburton

Agboh

et al. 2020

2020 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Humans, in comparison to robots, are remarkably adept at reaching for objects in cluttered environments. The best existing robot planners are based on random sampling in configuration space-which becomes excessively highdimensional with a large number of objects. Consequently, most of these planners suffer from limited object manipulation. We address this problem by learning high-level manipulation planning skills from humans and transfer these skills to robot planners. We used virtual reality to generate data from human participants whilst they reached for objects on a cluttered table top. From this, we devised a qualitative representation of the task space to abstract human decisions, irrespective of the number of objects in the way. Based on this representation, human demonstrations were segmented and used to train decision classifiers. Using these classifiers, our planner produced a list of waypoints in task space. These waypoints provide a high-level plan, which can be transferred to an arbitrary robot model and used to initialize a local trajectory optimiser. We evaluated this approach through testing on unseen human VR data, a physics-based robot simulation and real robot experiments. We find that this human-like planner outperforms a state-of-the-art standard trajectory optimisation algorithm and is able to generate effective strategies for rapid planning, irrespective of the number of objects in a cluttered environment. Our dataset and source code are publicly available 1 .

show abstract

“…The evolution of manipulation tasks is another wildly studied aspect that directly utilizes manipulation knowledge in robotics. Task evolution can be represented by structures such as semantic trees [6]- [9], state transition graphs [10], [11] or behavior trees [12], [13]. However, most of those evolution representations rely heavily on human annotations, and few of the aforementioned studies have discussed how to automatically acquire evolution representations on-scene.…”

Section: Related Work a Manipulation Knowledge In Roboticsmentioning

confidence: 99%

Understanding Contexts Inside Robot and Human Manipulation Tasks through a Vision-Language Model and Ontology System in a Video Stream

Jiang¹,

Dehghan²,

Jägersand³

2020

Preprint

View full text Add to dashboard Cite

Manipulation tasks in daily life, such as pouring water, unfold intentionally under specialized manipulation contexts. Being able to process contextual knowledge in these Activities of Daily Living (ADLs) over time can help us understand manipulation intentions, which are essential for an intelligent robot to transition smoothly between various manipulation actions. In this paper, to model the intended concepts of manipulation, we present a vision dataset under a strictly constrained knowledge domain for both robot and human manipulations, where manipulation concepts and relations are stored by an ontology system in a taxonomic manner. Furthermore, we propose a scheme to generate a combination of visual attentions and an evolving knowledge graph filled with commonsense knowledge. Our scheme works with real-world camera streams and fuses an attention-based Vision-Language model with the ontology system. The experimental results demonstrate that the proposed scheme can successfully represent the evolution of an intended object manipulation procedure for both robots and humans. The proposed scheme allows the robot to mimic human-like intentional behaviors by watching real-time videos. We aim to develop this scheme further for real-world robot intelligence in Human-Robot Interaction.

show abstract

Multi-Task Hierarchical Imitation Learning for Home Automation

Cited by 25 publications

References 22 publications

Continuous Control with Action Quantization from Demonstrations

Continuous Control with Action Quantization from Demonstrations

Human-like Planning for Reaching in Cluttered Environments

Understanding Contexts Inside Robot and Human Manipulation Tasks through a Vision-Language Model and Ontology System in a Video Stream

Contact Info

Product

Resources

About