Habitat: A Platform for Embodied AI Research

Savva, Manolis; Kadian, Abhishek; Maksymets, Oleksandr; Yan, Zhao; Wijmans, Erik; Jain, Bhavana; Straub, Julian; Liu, Jia; Koltun, Vladlen; Malik, Jitendra; Parikh, Devi; Batra, Dhruv

doi:10.48550/arxiv.1904.01201

Cited by 26 publications

(63 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We endow the dataset with the same structure as AVD by densely sampling navigable positions at 30cm intervals on the occupancy map of each scene. At each navigable position, we render RGB images, semantic segmentations and depth images from 12 different orientations at 30 • intervals through Habitat-Sim [20]. The images are connected through actions based on their spatial neighborhood and orientations.…”

Section: Methodsmentioning

confidence: 99%

Simultaneous Mapping and Target Driven Navigation

Georgakis,

Li,

Kosecka

2019

Preprint

View full text Add to dashboard Cite

This work presents a modular architecture for simultaneous mapping and target driven navigation in indoors environments. The semantic and appearance stored in 2.5D map is distilled from RGB images, semantic segmentation and outputs of object detectors by convolutional neural networks. Given this representation, the mapping module learns to localize the agent and register consecutive observations in the map. The navigation task is then formulated as a problem of learning a policy for reaching semantic targets using current observations and the up-to-date map. We demonstrate that the use of semantic information improves localization accuracy and the ability of storing spatial semantic map aids the target driven navigation policy. The two modules are evaluated separately and jointly on Active Vision Dataset [1] and Matterport3D environments [3], demonstrating improved performance on both localization and navigation tasks.

show abstract

Section: Methodsmentioning

confidence: 99%

Simultaneous Mapping and Target Driven Navigation

Georgakis,

Li,

Kosecka

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…Publicly available simulated environments are playing an important role in the development of RL methods, provide a common ground for comparing different approaches, and allow to track the progress of the field. Simulated environments address various general aspects of reinforcement learning research such as control [48], navigation [50], [51], [52], [53], physical interactions [49] and perception [54]. More domain-specific environments explore such fields as robotics [55], [56], [57] and autonomous driving [58].…”

Section: Related Workmentioning

confidence: 99%

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

Kumar¹,

Buckley²,

Lanier³

et al. 2019

Preprint

View full text Add to dashboard Cite

Success stories of applied machine learning can be traced back to the datasets and environments that were put forward as challenges for the community. The challenge that the community sets as a benchmark is usually the challenge that the community eventually solves. The ultimate challenge of reinforcement learning research is to train real agents to operate in the real environment, but until now there has not been a common real-world RL benchmark. In this work, we present a prototype real-world environment from OffWorld Gym -a collection of real-world environments for reinforcement learning in robotics with free public remote access. Close integration into existing ecosystem allows the community to start using OffWorld Gym without any prior experience in robotics and takes away the burden of managing a physical robotics system, abstracting it under a familiar API. We introduce a navigation task, where a robot has to reach a visual beacon on an uneven terrain using only the camera input and provide baseline results in both the real environment and the simulated replica. To start training, visit https://gym.offworld.ai.

show abstract

“…Extensive work has demonstrated that, after learning with indirect supervision from a reward function, rich representations for their task automatically emerge (Bansal et al, 2017;Lowe et al, 2017;Jaderberg et al, 2019). Several recent works have created 3D embodiment simulation environment (Kolve et al, 2017;Brodeur et al, 2017;Savva et al, 2017;Das et al, 2018;Xia et al, 2018;Savva et al, 2019) for navigation and visual question answering tasks. To train these models, visual navigation is often framed as a reinforcement learning problem (Chen et al, 2015;Giusti et al, 2015;Oh et al, 2016;Abel et al, 2016;Bhatti et al, 2016;Daftry et al, 2016;Mirowski et al, 2016;Brahmbhatt & Hays, 2017;Zhang et al, 2017a;Zhu et al, 2017a;Kahn et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Visual Hide and Seek

Chen¹,

Song²,

Lipson³

et al. 2019

Preprint

View full text Add to dashboard Cite

We train embodied agents to play Visual Hide and Seek where a prey must navigate in a simulated environment in order to avoid capture from a predator. We place a variety of obstacles in the environment for the prey to hide behind, and we only give the agents partial observations of their environment using an egocentric perspective. Although we train the model to play this game from scratch, experiments and visualizations suggest that the agent learns to predict its own visibility in the environment. Furthermore, we quantitatively analyze how agent weaknesses, such as slower speed, effect the learned policy. Our results suggest that, although agent weaknesses make the learning problem more challenging, they also cause more useful features to be learned.

show abstract

Habitat: A Platform for Embodied AI Research

Cited by 26 publications

References 0 publications

Simultaneous Mapping and Target Driven Navigation

Simultaneous Mapping and Target Driven Navigation

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

Visual Hide and Seek

Contact Info

Product

Resources

About