A survey on large language model based autonomous agents

Abstract

Autonomous agents have long been a research focus in academic and industry communities. Previous research often focuses on training agents with limited knowledge within isolated environments, which diverges significantly from human learning processes, and makes the agents hard to achieve human-like decisions. Recently, through the acquisition of vast amounts of Web knowledge, large language models (LLMs) have shown potential in human-level intelligence, leading to a surge in research on LLM-based autonomous agents. In this paper, we present a comprehensive survey of these studies, delivering a systematic review of LLM-based autonomous agents from a holistic perspective. We first discuss the construction of LLM-based autonomous agents, proposing a unified framework that encompasses much of previous work. Then, we present a overview of the diverse applications of LLM-based autonomous agents in social science, natural science, and engineering. Finally, we delve into the evaluation strategies commonly used for LLM-based autonomous agents. Based on the previous studies, we also present several challenges and future directions in this field.

Recommendations for designing conversational companion robots with older adults through foundation models

“…To alleviate loneliness among older adults, companion robots can provide users with the opportunity to reconnect with friends and family, thereby, mitigating the risks of over-reliance on interaction with technology. Foundation models capable of utilizing tools for social media, phones, and various devices (see Wang et al (2024) for a survey) that leverage edge computing can enable this functionality (e.g., Dong L. et al, 2023;Shen et al, 2023). Additionally, robots can facilitate new online connections for users by harnessing their social media networks with the assistance of other deep learning architectures (e.g., Ding et al (2017); Chen et al (2020).…”

“…Semantic understanding, i.e., the relations among entities within visual scenes through object, scene, or action recognition, can be achieved with foundation models to provide advice based on the situational context that extends beyond the capabilities of verbal context ( Bommasani et al, 2022 ). For instance, the robot can suggest the user a recipe based on their preferences, and offer help with cooking verbally or potentially physically if integrated with manipulators, in which foundation models can be used for generating robot plans and actions, by referring to/using the learned locations of the equipment and ingredients (see Wang et al (2024) ; Firoozi et al (2023) for surveys of LLMs and foundation models in robotics for task planning and control).…”

A Data-Intelligence-Intensive Bioinformatics Copilot System for Large-scale Omics Researches and Scientific Insights

“…In recent years, the revolutionary advancements in deep learning technologies, highlighted by the introduction of LLMs such as the GPT series, have empowered LLM-based agents with formidable natural language processing capabilities. Agents 41,42,43,98 , capable of autonomously perceiving their environment, cognitive reasoning, decision-making, and executing actions through tool invocation, have emerged as a highly promising direction in the pursuit of general artificial intelligence. Specifically, an AI agent comprises four modules: the perception module for gathering environmental information, the cognition and decision-making module to analyze inputs and devise action strategies, the memory module to archive knowledge and past behaviors, and the action module to implement decisions by manipulating tools to impact the environment.…”

“…1b. Agents, the core components of DII-MAS, employ LLMs as their controlling nucleus and exhibit a high degree of autonomous capability 41,42,43 .…”

“…1b. Agents, the core components of DII-MAS, employ LLMs as their controlling nucleus and exhibit a high degree of autonomous capability 41,42,43 . They are capable of perceiving environmental information, deciphering complex natural language instructions, conducting cognitive reasoning, and executing actions through tool invocations.…”