Out of the Box: Embodied Navigation in the Real World

Bigazzi, Roberto; Landi, Federico; Cornia, Marcella; Cascianelli, Silvia; Baraldi, Lorenzo; Cucchiara, Rita

doi:10.1007/978-3-030-89128-2_5

Cited by 10 publications

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Those agents are commonly trained with reinforcement learning adopting a modular, hierarchical approach [10] where the agent learns to explore the environment by optimizing a self-supervision signal in the form of a reward function [10,29,30,4]. Once trained on the aforementioned simulators, the developed agents can be easily deployed on physical robotic platforms [6]. However, many state-of-the-art architectures are still considered black boxes, as their behavior lacks explainability [2].…”

Section: Related Workmentioning

confidence: 99%

Towards Explainable Navigation and Recounting

Poppi,

Bigazzi,

Rawal

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Explainable Navigation and Recounting

Poppi,

Bigazzi,

Rawal

et al. 2023

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…Both autonomous robotics [5,14] and embodied AI [8,11,18,6,12,21] have recently witnessed a boost of interest, which has been enabled by the release of photorealistic 3D simulated environments. In such environments, algorithms for intelligent exploration and navigation can be developed safely and more quickly than in the real-world, before being easily deployed on real robotic platforms [15,7,2]. Among the datasets of spaces, the most commonly used are MP3D [9], Gibson [26], HM3D [22], and Replica [25].…”

Section: Related Workmentioning

confidence: 99%

“…Hence, models trained to explore the Gibson dataset can solve Point-Goal navigation with satisfactory accuracy under the appropriate hypotheses. Furthermore, accurate and realistic simulating platforms such as Habitat [23] facilitate the deployment in the real world of the trained agents [7,15]. While agent architectures and simulating platforms are possible sources of improvement, there is a third important direction of research that regards the availability of 3D scenes to train and test the different agents.…”

Section: Introductionmentioning

confidence: 99%

Embodied Navigation at the Art Gallery

Bigazzi¹,

Landi²,

Cascianelli³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Embodied agents, trained to explore and navigate indoor photorealistic environments, have achieved impressive results on standard datasets and benchmarks. So far, experiments and evaluations have involved domestic and working scenes like offices, flats, and houses. In this paper, we build and release a new 3D space with unique characteristics: the one of a complete art museum. We name this environment ArtGallery3D (AG3D). Compared with existing 3D scenes, the collected space is ampler, richer in visual features, and provides very sparse occupancy information. This feature is challenging for occupancy-based agents which are usually trained in crowded domestic environments with plenty of occupancy information. Additionally, we annotate the coordinates of the main points of interest inside the museum, such as paintings, statues, and other items. Thanks to this manual process, we deliver a new benchmark for PointGoal navigation inside this new space. Trajectories in this dataset are far more complex and lengthy than existing groundtruth paths for navigation in Gibson and Matterport3D. We carry on extensive experimental evaluation using our new space for evaluation and prove that existing methods hardly adapt to this scenario. As such, we believe that the availability of this 3D model will foster future research and help improve existing solutions.

show abstract

“…In robotics, simulation environments have been developed to quickly and safely train policies, with the eventual goal of transferring them to real world applications [42,74,44,7,84,66,60,72]. The resulting sim-to-real problem, where models must adapt to changes between simulator and real-world domains, is an active area of research [73,41,65,4,1]. Simulated data for video understanding is much less explored.…”

Section: Related Workmentioning

confidence: 99%

Egocentric scene context for human-centric environment understanding from video

Nagarajan¹,

Ramakrishnan²,

Desai³

et al. 2022

Preprint

View full text Add to dashboard Cite

First-person video highlights a camera-wearer's activities in the context of their persistent environment. However, current video understanding approaches reason over visual features from short video clips that are detached from the underlying physical space and only capture what is directly seen. We present an approach that links egocentric video and camera pose over time by learning representations that are predictive of the camera-wearer's (potentially unseen) local surroundings to facilitate human-centric environment understanding. We train such models using videos from agents in simulated 3D environments where the environment is fully observable, and test them on real-world videos of house tours from unseen environments. We show that by grounding videos in their physical environment, our models surpass traditional scene classification models at predicting which room a camera-wearer is in (where frame-level information is insufficient), and can leverage this grounding to localize video moments corresponding to environment-centric queries, outperforming prior methods. Project page: http://vision.cs.utexas. edu/projects/ego-scene-context/

show abstract

Out of the Box: Embodied Navigation in the Real World

Cited by 10 publications

References 9 publications

Towards Explainable Navigation and Recounting

Towards Explainable Navigation and Recounting

Embodied Navigation at the Art Gallery

Egocentric scene context for human-centric environment understanding from video

Contact Info

Product

Resources

About