Expressing homotopic requirements for mobile robot navigation through natural language instructions

Yi, Daqing; Howard, Thomas M.; Goodrich, Michael A.; Seppi, Kevin

doi:10.1109/iros.2016.7759238

Cited by 6 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Observing that aspects of the ADCG and HDCG could be combined to provide even more efficient approximations of the symbolic representation, the HADCG (Paul et al, 2018) combined these ideas into an approach that demonstrated an improvement in the efficiency of language grounding without a loss of accuracy over the DCG, HDCG, and ADCG in a series of experiments involving manipulation and navigation commands. Variations of these models with domain specific symbolic representations have been applied for applications involving language grounding for synthesis of verifiable controllers (Boteanu et al, 2016(Boteanu et al, , 2017, adaptive grasp control (Esponda and Howard, 2018), planning corrections for assistive robotic manipulators (Broad et al, 2017), and homotopy-aware motion planning (Yi et al, 2016).…”

Section: Natural Language Understandingmentioning

confidence: 99%

An Intelligence Architecture for Grounded Language Communication with Field Robots

Howard¹,

Stump²,

Fink³

et al. 2022

View full text Add to dashboard Cite

For humans and robots to collaborate effectively as teammates in unstructured environments, robots must be able to construct semantically rich models of the environment, communicate efficiently with teammates, and perform sequences of tasks robustly with minimal human intervention, as direct human guidance may be infrequent and/or intermittent. Contemporary architectures for human-robot interaction often rely on engineered human-interface devices or structured languages that require extensive prior training and inherently limit the kinds of information that humans and robots can communicate. Natural language, particularly when situated with a visual representation of the robot’s environment, allows humans and robots to exchange information about abstract goals, specific actions, and/or properties of the environment quickly and effectively. In addition, it serves as a mechanism to resolve inconsistencies in the mental models of the environment across the human-robot team. This article details a novel intelligence architecture that exploits a centralized representation of the environment to perform complex tasks in unstructured environments. The centralized environment model is informed by a visual perception pipeline, declarative knowledge, deliberate interactive estimation, and a multimodal interface. The language pipeline also exploits proactive symbol grounding to resolve uncertainty in ambiguous statements through inverse semantics. A series of experiments on three different, unmanned ground vehicles demonstrates the utility of this architecture through its robust ability to perform language-guided spatial navigation, mobile manipulation, and bidirectional communication with human operators. Experimental results give examples of component-level behaviors and overall system performance that guide a discussion on observed performance and opportunities for future innovation.

show abstract

Section: Natural Language Understandingmentioning

confidence: 99%

An Intelligence Architecture for Grounded Language Communication with Field Robots

Howard¹,

Stump²,

Fink³

et al. 2022

View full text Add to dashboard Cite

show abstract

“…There is a long history of robots and humans sharing spatial descriptions, going back to SRI's Shakey in the 1960s arXiv:1903.03669v1 [cs.RO] 8 Mar 2019 and 1970s [13]. There is a large body of work on robots that can understand natural human commands [19,21,23], while fewer studies have focused on robots that can translate their observations to descriptions or instructions. In work explicitly considering human-robot interaction systems, Skubic et al [18] investigated spatial semantic models for human-robot dialogue where the robot describes the spatial relation of objects with respect to itself.…”

Section: Related Workmentioning

confidence: 99%

Recognizing and Tracking High-Level, Human-Meaningful Navigation Features of Occupancy Grid Maps

Nikdel

Chen

2020

2020 17th Conference on Computer and Robot Vision (CRV)

View full text Add to dashboard Cite

This paper describes a system whereby a robot detects and track human-meaningful navigational cues as it navigates in an indoor environment. It is intended as the sensor front-end for a mobile robot system that can communicate its navigational context with human users. From simulated LiDAR scan data we construct a set of 2D occupancy grid bitmaps, then hand-label these with human-scale navigational features such as closed doors, open corridors and intersections. We train a Convolutional Neural Network (CNN) to recognize these features on input bitmaps. In our demonstration system, these features are detected at every time step then passed to a tracking module that does frame-to-frame data association to improve detection accuracy and identify stable unique features. We evaluate the system in both simulation and the real world. We compare the performance of using input occupancy grids obtained directly from LiDAR data, or incrementally constructed with SLAM, and their combination.

show abstract

“…Previous work has grounded natural language navigational commands to executable representations. Graphical modelbased approaches using syntactic parses have been applied to controlling robotic forklift actions [4] and mobile navigation in novel environments [5], [6]. Others have utilized CCG semantic parsing of robotic commands in synthetic environments [7], and with weak supervision [8].…”

Section: Related Workmentioning

confidence: 99%

Balancing Shared Autonomy with Human-Robot Communication

Scalise¹,

Bisk²,

Forbes³

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

Robotic agents that share autonomy with a human should leverage human domain knowledge and account for their preferences when completing a task. This extra knowledge can dramatically improve plan efficiency and user-satisfaction, but these gains are lost if communicating with a robot is taxing and unnatural. In this paper, we show how viewing humanrobot language through the lens of shared autonomy explains the efficiency versus cognitive load trade-offs humans make when deciding how cooperative and explicit to make their instructions.1 The planner performs a depth-first search over a permutation graph of manipulable objects until it finds a geometrically feasible solution which addresses all objects.

show abstract

Expressing homotopic requirements for mobile robot navigation through natural language instructions

Cited by 6 publications

References 17 publications

An Intelligence Architecture for Grounded Language Communication with Field Robots

An Intelligence Architecture for Grounded Language Communication with Field Robots

Recognizing and Tracking High-Level, Human-Meaningful Navigation Features of Occupancy Grid Maps

Balancing Shared Autonomy with Human-Robot Communication

Contact Info

Product

Resources

About