Learning Human Search Behavior from Egocentric Visual Inputs

Sorokin, Maks; Yu, Wenhao; Ha, Sehoon; Liu, C. Karen

doi:10.1111/cgf.142641

Cited by 3 publications

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Learning to Navigate Sidewalks in Outdoor Environments

Sorokin

Tan

Liu

et al. 2022

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

As robots become more prevalent, optimizing their design for better performance and efficiency is becoming increasingly important. However, current robot design practices overlook the impact of perception and design choices on a robot's learning capabilities. To address this gap, we propose a comprehensive methodology that accounts for the interplay between the robot's perception, hardware characteristics, and task requirements. Our approach optimizes the robot's morphology holistically, leading to improved learning and task execution proficiency. To achieve this, we introduce a Morphology-AGnostIc Controller (MAGIC), which helps with the rapid assessment of different robot designs. The MAGIC policy is efficiently trained through a novel PRIvileged Single-stage learning via latent alignMent (PRISM) framework, which also encourages behaviors that are typical of robot onboard observation. Our simulation-based results demonstrate that morphologies optimized holistically improve the robot performance by 15-20% on various manipulation tasks, and require 25x less data to match human-expert made morphology performance. In summary, our work contributes to the growing trend of learning-based approaches in robotics and emphasizes the potential in designing robots that facilitate better learning. The project's website can be found at learning-robot.github.io

show abstract

Learning to Navigate Sidewalks in Outdoor Environments

Sorokin

Tan

Liu

et al. 2022

IEEE Robot. Autom. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

Cascaded Compositional Residual Learning for Complex Interactive Behaviors

Kumar

Essa

2023

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Learning to Navigate Sidewalks in Outdoor Environments

Sorokin

Tan

Liu

et al. 2021

Preprint

View full text Add to dashboard Cite

Outdoor navigation on sidewalks in urban environments is the key technology behind important human assistive applications, such as last-mile delivery or neighborhood patrol. This paper aims to develop a quadruped robot that follows a route plan generated by public map services, while remaining on sidewalks and avoiding collisions with obstacles and pedestrians. We devise a two-staged learning framework, which first trains a teacher agent in an abstract world with privileged ground-truth information, and then applies Behavior Cloning to teach the skills to a student agent who only has access to realistic sensors. The main research effort of this paper focuses on overcoming challenges when deploying the student policy on a quadruped robot in the real world. We propose methodologies for designing sensing modalities, network architectures, and training procedures to enable zeroshot policy transfer to unstructured and dynamic real outdoor environments. We evaluate our learning framework on a quadrupedal robot navigating sidewalks in the city of Atlanta, USA. Using the learned navigation policy and its onboard sensors, the robot is able to walk 3.2 kilometers with a limited number of human interventions.

show abstract

Learning Human Search Behavior from Egocentric Visual Inputs

Abstract: Figure 1: A humanoid character learns to navigate and search for a target object (the mustard bottle) in a photorealistic 3D scene using its own egocentric vision and locomotion capability. Top: third-person view. Bottom: first-person view.

Cited by 3 publications

References 41 publications

Learning to Navigate Sidewalks in Outdoor Environments

Learning to Navigate Sidewalks in Outdoor Environments

Cascaded Compositional Residual Learning for Complex Interactive Behaviors

Learning to Navigate Sidewalks in Outdoor Environments

Contact Info

Product

Resources

About