LLM-Informed Multi-Armed Bandit Strategies for Non-Stationary Environments

Curtò, J. de; Zarzà, I. de; Roig, Gemma; Cano, Juan-Carlos; Manzoni, Pietro; Calafate, Carlos T.

doi:10.3390/electronics12132814

Cited by 7 publications

(4 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We also note that some related studies have applied MAS (multi-agent systems) [33][34][35], large language models (LLMs) [36][37][38], and visual language models (VLMs) [39] to robot navigation and guidance. They utilize sensors like cameras, laser scanners, or radar to gather detailed environmental data.…”

Section: Discussionmentioning

confidence: 99%

Pursuit Problem of Unmanned Aerial Vehicles

Oleg,

Zhang

2023

Mathematics

View full text Add to dashboard Cite

The study examines scenarios involving a single pursuer tracking a single evader, as well as situations where multiple pursuers are involved in chasing multiple evaders. We formulate this problem as a search and pursuit problem for unmanned aerial vehicles (UAVs). Game theory offers a mathematical framework to model and examine strategic interactions involving multiple decision-makers. By employing game theory principles to address the search and pursuit problem, our objective is to optimize the efficiency of strategies for detecting and capturing unmanned aerial vehicles (UAVs).

show abstract

Section: Discussionmentioning

confidence: 99%

Pursuit Problem of Unmanned Aerial Vehicles

Oleg,

Zhang

2023

Mathematics

View full text Add to dashboard Cite

show abstract

“…The usage of AI, and more specifically LLMs [11,12], in the scientific field has seen a surge in recent years. OpenAI's GPT-3, the predecessor to GPT-3.5 Turbo, has been utilized in various scientific domains [16][17][18][19]. These studies highlight the capability of LLMs to generate informative, contextually relevant content, and suggest the potential for their application in more specialized scientific tasks [20][21][22].…”

Section: Related Workmentioning

confidence: 99%

Large Language Model-Informed X-ray Photoelectron Spectroscopy Data Analysis

de Curtò,

de Zarzà,

Roig

et al. 2024

Signals

Self Cite

View full text Add to dashboard Cite

X-ray photoelectron spectroscopy (XPS) remains a fundamental technique in materials science, offering invaluable insights into the chemical states and electronic structure of a material. However, the interpretation of XPS spectra can be complex, requiring deep expertise and often sophisticated curve-fitting methods. In this study, we present a novel approach to the analysis of XPS data, integrating the utilization of large language models (LLMs), specifically OpenAI’s GPT-3.5/4 Turbo to provide insightful guidance during the data analysis process. Working in the framework of the CIRCE-NAPP beamline at the CELLS ALBA Synchrotron facility where data are obtained using ambient pressure X-ray photoelectron spectroscopy (APXPS), we implement robust curve-fitting techniques on APXPS spectra, highlighting complex cases including overlapping peaks, diverse chemical states, and noise presence. Post curve fitting, we engage the LLM to facilitate the interpretation of the fitted parameters, leaning on its extensive training data to simulate an interaction corresponding to expert consultation. The manuscript presents also a real use case utilizing GPT-4 and Meta’s LLaMA-2 and describes the integration of the functionality into the TANGO control system. Our methodology not only offers a fresh perspective on XPS data analysis, but also introduces a new dimension of artificial intelligence (AI) integration into scientific research. It showcases the power of LLMs in enhancing the interpretative process, particularly in scenarios wherein expert knowledge may not be immediately available. Despite the inherent limitations of LLMs, their potential in the realm of materials science research is promising, opening doors to a future wherein AI assists in the transformation of raw data into meaningful scientific knowledge.

show abstract

“…Ref. [28] propose an LLM-based strategy that enables adaptive balancing of exploration and exploitation. Ref.…”

Section: Related Workmentioning

confidence: 99%

Prompt Optimization in Large Language Models

Sabbatella,

Ponti,

Giordani

et al. 2024

Mathematics

View full text Add to dashboard Cite

Prompt optimization is a crucial task for improving the performance of large language models for downstream tasks. In this paper, a prompt is a sequence of n-grams selected from a vocabulary. Consequently, the aim is to select the optimal prompt concerning a certain performance metric. Prompt optimization can be considered as a combinatorial optimization problem, with the number of possible prompts (i.e., the combinatorial search space) given by the size of the vocabulary (i.e., all the possible n-grams) raised to the power of the length of the prompt. Exhaustive search is impractical; thus, an efficient search strategy is needed. We propose a Bayesian Optimization method performed over a continuous relaxation of the combinatorial search space. Bayesian Optimization is the dominant approach in black-box optimization for its sample efficiency, along with its modular structure and versatility. We use BoTorch, a library for Bayesian Optimization research built on top of PyTorch. Specifically, we focus on Hard Prompt Tuning, which directly searches for an optimal prompt to be added to the text input without requiring access to the Large Language Model, using it as a black-box (such as for GPT-4 which is available as a Model as a Service). Albeit preliminary and based on “vanilla” Bayesian Optimization algorithms, our experiments with RoBERTa as a large language model, on six benchmark datasets, show good performances when compared against other state-of-the-art black-box prompt optimization methods and enable an analysis of the trade-off between the size of the search space, accuracy, and wall-clock time.

show abstract

LLM-Informed Multi-Armed Bandit Strategies for Non-Stationary Environments

Cited by 7 publications

References 9 publications

Pursuit Problem of Unmanned Aerial Vehicles

Pursuit Problem of Unmanned Aerial Vehicles

Large Language Model-Informed X-ray Photoelectron Spectroscopy Data Analysis

Prompt Optimization in Large Language Models

Contact Info

Product

Resources

About