LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

Song, Chan Hee; Wu, Jiaman; Washington, Clayton; Sadler, Brian M.; Chao, Weilun; Sui, Yu

doi:10.48550/arxiv.2212.04088

Cited by 9 publications

(15 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Introspective Tips [17] allows LLM to introspect through the history of the environmental feedback. LLM-Planner [127] introduces a grounded re-planning algorithm that dynamically updates plans generated by LLMs when encountering object mismatches and unattainable plans during task completion. In Progprompt [126], assertions are incorporated into the generated script to provide environment state feedback, allowing for error recovery in case the action's preconditions are not satisfied.…”

Section: Planning Modulementioning

confidence: 99%

Genomic Surveillance for SARS-CoV-2 — China, September 26, 2022 to January 29, 2023

Wang¹,

Niu²,

Su³

et al. 2023

China CDC Weekly

View full text Add to dashboard Cite

Introduction: The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has generated 2,431 variants over the course of its global transmission over the past 3 years. To better evaluate the genomic variation of SARS-CoV-2 before and after the optimization of coronavirus disease 2019 (COVID-19) prevention and control strategies, we analyzed the genetic evolution branch composition and genomic variation of SARS-CoV-2 in both domestic and imported cases in China (the data from Hong Kong and Macau Special Administrativc Regions and Taiwan, China were not included) from September 26, 2022 to January 29, 2023.Methods: Analysis of the number of genome sequences, sampling time, dynamic changes of evolutionary branches, origin, and clinical typing of SARS-CoV-2 variants submitted by 31 provincial-level administrative divisions (PLADs) and Xinjiang Production and Construction Corp (XPCC) was conducted to assess the accuracy and timeliness of SARS-CoV-2 variant surveillance.Results: From September 26, 2022 to January 29, 2023, 20,013 valid genome sequences of domestic cases were reported in China, with 72 evolutionary branches. Additionally, 1,978 valid genome sequences of imported cases were reported, with 169 evolutionary branches. The prevalence of the Omicron variants of SARS-CoV-2 in both domestic and imported cases was consistent with that of international epidemic variants.Conclusions: This study provides an overview of the prevalence of Omicron variants of SARS-CoV-2 in China. After optimizing COVID-19 prevention and control strategies, no novel Omicron variants of SARS-CoV-2 with altered biological characteristics or public health significance have been identified since

show abstract

Section: Planning Modulementioning

confidence: 99%

Genomic Surveillance for SARS-CoV-2 — China, September 26, 2022 to January 29, 2023

Wang¹,

Niu²,

Su³

et al. 2023

China CDC Weekly

View full text Add to dashboard Cite

show abstract

“…Large language models have been explored as an approach to high-level planning [14]- [18] and scene understanding [19], [20]. Vision-language models embedding image features into the same space as text have been applied to open vocabulary object detection [16], [17], natural language maps [15], [17], [21]- [23], and for language-informed navigation [24]- [26].…”

Section: Language Models In Roboticsmentioning

confidence: 99%

Neural Implicit Vision-Language Feature Fields

Blomqvist¹,

Milano²,

Chung³

et al. 2023

Preprint

View full text Add to dashboard Cite

Recently, groundbreaking results have been presented on open-vocabulary semantic image segmentation. Such methods segment each pixel in an image into arbitrary categories provided at run-time in the form of text prompts, as opposed to a fixed set of classes defined at training time. In this work, we present a zero-shot volumetric open-vocabulary semantic scene segmentation method. Our method builds on the insight that we can fuse image features from a visionlanguage model into a neural implicit representation. We show that the resulting feature field can be segmented into different classes by assigning points to natural language text prompts. The implicit volumetric representation enables us to segment the scene both in 3D and 2D by rendering feature maps from any given viewpoint of the scene. We show that our method works on noisy real-world data and can run in real-time on live sensor data dynamically adjusting to text prompts. We also present quantitative comparisons on the ScanNet dataset.

show abstract

“…The prompt instructs the LLM on what actions to generate and only permits using the provided objects when generating the plan. Song et al [32] improve on that general idea by utilizing dynamic in-context example retrieval to enhance performance and enable the LLM to replan in the event of an error. The authors of [29] reduce problem complexity by not giving the LLM all objects but filtering out irrelevant objects by traversing a 3D scene graph via collapsing and expanding notes before letting the LLM output a plan.…”

Section: B Large Language Models In Planningmentioning

confidence: 99%

Oriented Surface Reachability Maps for Robot Placement

Birr

Pohl

Asfour

2022

2022 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Recent advances in task planning leverage Large Language Models (LLMs) to improve generalizability by combining such models with classical planning algorithms to address their inherent limitations in reasoning capabilities. However, these approaches face the challenge of dynamically capturing the initial state of the task planning problem. To alleviate this issue, we propose AutoGPT+P, a system that combines an affordance-based scene representation with a planning system. Affordances encompass the action possibilities of an agent on the environment and objects present in it. Thus, deriving the planning domain from an affordance-based scene representation allows symbolic planning with arbitrary objects. AutoGPT+P leverages this representation to derive and execute a plan for a task specified by the user in natural language. In addition to solving planning tasks under a closed-world assumption, AutoGPT+P can also handle planning with incomplete information, e. g., tasks with missing objects by exploring the scene, suggesting alternatives, or providing a partial plan. The affordance-based scene representation combines object detection with an automatically generated object-affordance-mapping using ChatGPT. The core planning tool extends existing work by automatically correcting semantic and syntactic errors. Our approach achieves a success rate of 98%, surpassing the current 81% success rate of the current state-of-the-art LLM-based planning method SayCan on the SayCan instruction set. Furthermore, we evaluated our approach on our newly created dataset with 150 scenarios covering a wide range of complex tasks with missing objects, achieving a success rate of 79% on our dataset. The dataset and the code are publicly available at https://git.h2t.iar.kit.edu/birr/autogpt-p-standalone.

show abstract

LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

Cited by 9 publications

References 0 publications

Genomic Surveillance for SARS-CoV-2 — China, September 26, 2022 to January 29, 2023

Genomic Surveillance for SARS-CoV-2 — China, September 26, 2022 to January 29, 2023

Neural Implicit Vision-Language Feature Fields

Oriented Surface Reachability Maps for Robot Placement

Contact Info

Product

Resources

About