Instruction completion through instance-based learning and semantic analogical reasoning

Nyga, Daniel; Picklum, Mareike; Koralewski, Sebastian; Beetz, Michael

doi:10.1109/icra.2017.7989491

Cited by 9 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior work in open-world planning has developed techniques to make robots programmed by expert users more robust to environment variation through the use of commonsense or domain knowledge [19], [20], [21], [22], [23]. Open-world planners can achieve plan goals in environments with variations [19], under-specified plans [21], [24], and partially observed worlds [20] by coupling a knowledge base with declarative programming planners, such as PDDL [19], CRAM [21], or ASP [20]. However, these techniques tend to depend on hand engineered domain knowledge inputs that end-users cannot provide in a new demonstration, such as a taxonomy of relevant concepts or an ontology of the domain.…”

Section: Related Workmentioning

confidence: 99%

Towards Robust One-shot Task Execution using Knowledge Graph Embeddings

Daruna¹,

Nair²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Requiring multiple demonstrations of a task plan presents a burden to end-users of robots. However, robustly executing tasks plans from a single end-user demonstration is an ongoing challenge in robotics. We address the problem of one-shot task execution, in which a robot must generalize a single demonstration or prototypical example of a task plan to a new execution environment. Our approach integrates task plans with domain knowledge to infer task plan constituents for new execution environments. Our experimental evaluations show that our knowledge representation makes more relevant generalizations that result in significantly higher success rates over tested baselines. We validated the approach on a physical platform, which resulted in the successful generalization of initial task plans to 38 of 50 execution environments with errors resulting from autonomous robot operation included.

show abstract

Section: Related Workmentioning

confidence: 99%

Towards Robust One-shot Task Execution using Knowledge Graph Embeddings

Daruna¹,

Nair²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most of these methods are limited to navigation applications [12], [11], [6] or simple settings [7], or they are not evaluated on real robots [10]. Nyga et al [26], [27], [28], [29] use probabilistic relation models based on knowledge bases to understand natural language commands that describe visual attributes of objects. This is complementary to our work.…”

Section: Benefits and Comparisonsmentioning

confidence: 99%

Efficient Generation of Motion Plans from Attribute-Based Natural Language Instructions Using Dynamic Constraint Mapping

Park

Jia²,

Bansal³

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We present an algorithm for combining natural language processing (NLP) and fast robot motion planning to automatically generate robot movements. Our formulation uses a novel concept called Dynamic Constraint Mapping to transform complex, attribute-based natural language instructions into appropriate cost functions and parametric constraints for optimization-based motion planning. We generate a factor graph from natural language instructions called the Dynamic Grounding Graph (DGG), which takes latent parameters into account. The coefficients of this factor graph are learned based on conditional random fields (CRFs) and are used to dynamically generate the constraints for motion planning. We map the cost function directly to the motion parameters of the planner and compute smooth trajectories in dynamic scenes. We highlight the performance of our approach in a simulated environment and via a human interacting with a 7-DOF Fetch robot using intricate language commands including negation, orientation specification, and distance constraints.

show abstract