An adaptive-learning framework for semi-cooperative multi-agent coordination

Boukhtouta, Abdeslem; Berger, Jean; Powell, Warren B.; George, Abraham

doi:10.1109/adprl.2011.5967386

Cited by 7 publications

(7 citation statements)

References 67 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It required 5s to compute the new gain (batch solution), and this time is used to fill the matrices Γ and δ of Eq. (15)(16), necessary for the calculation of Φ k i and y k i in Eq. ( 12).…”

Section: A Drone Model In Z-axis and Numerical Simulationmentioning

confidence: 99%

“…A promising solution to solve the online control problem with plant uncertainties is the adaptive dynamic programming (ADP) method [14], [15]. This approach employs a 'forward-in-time' mechanism that looks for an optimal control policy by successively adapting two parametric structures, i.e., an action network and a critic network, to approximate the solution of the Hamilton-Jacobi-Bellman (HJB) equation [16], [17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

HDP Algorithms for Trajectory Tracking and Formation Control of Multi-Agent Systems

2022

View full text Add to dashboard Cite

A heuristic dynamic programming (HDP) algorithm for trajectory tracking and formation control of multi-agent systems (MAS) is presented in this paper. The selected HDP method allows for an online optimal control design. The multi-agent control problem is formulated as a leader-follower, where it is necessary for followers to maintain the assigned formation, while the leader follows the specified trajectory. Developed QR-Solver and RLS µ -QR-HDP-DLQR algorithms provide a new methodology to obtain the solution of the Hamilton-Jacobi-Bellman (HJB) equation. These proposed solutions are based on QR factorization to decrease the computational cost and avoid issues associated with numerical stability of conventional least squares (LS) method. The algorithms' performances such as convergence, numerical stability, and computational metrics, were experimentally evaluated for two control systems: aerial altitude control (one degree of freedom) and terrestrial robot (two degrees of freedom).

show abstract

Section: A Drone Model In Z-axis and Numerical Simulationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

HDP Algorithms for Trajectory Tracking and Formation Control of Multi-Agent Systems

2022

View full text Add to dashboard Cite

show abstract

“…In the work of Boukhtouta et al, resource allocation techniques are utilized to activate and optimize the activities of agents. In the approach, to achieve the common objective of maximizing demand coverage of the system, agents can share information regarding the future value of resources, the demands of agents on resources, the local decisions of agents, and the corresponding contributions or impacts on resources to find the best decisions.…”

Section: Related Workmentioning

confidence: 99%

A values‐driven self‐organization mechanism for automating multiagent coordination

Jiao

Sun

2018

Computational Intelligence

View full text Add to dashboard Cite

In distributed and open environments, MASs (multiagent systems) generally have no mechanisms for prior coordination and self‐organization has been believed to be the necessary selection to achieve the coordination of agents. This paper first presents a values‐driven model for self‐organization in which the expected emergent properties of a system are specified as the social values while the social values are realized via implicitly inducing members to regulate their individual values and adjust their behaviors to fit the expectations of the system. Based on the values‐driven self‐organization, this paper proposes an automated coordination mechanism for decentralized MASs. In this mechanism, by indirectly changing the difficulties in acquiring resources (which may be delegated to some special agents since MASs generally do not have substantial bodies), MASs can lead agents to regulate their values to be consistent with the social values of MASs so that the coordination of MASs can spontaneously emerge from the local behaviors of agents. Finally, this paper implements a simulation traffic system using the coordination mechanism based on values‐driven self‐organization to validate the emergence of coordination among multiple agents.

show abstract

“…Our behavioral analytics framework is also related to research that explores stochastic control of multi-agent systems. Related methods include decentralized control (Li et al 2012), approximate dynamic programming (Boukhtouta et al 2011, George andPowell 2007), game-theoretic approaches (Adlakha and Johari 2013, Iyer et al 2011, Zhou et al 2016, and robust optimization (Blanchet et al 2013, Bertsimas and Goyal 2012, Lorca and Sun 2015. In general, these models consider very different settings from the ones we consider in this paper.…”

Section: Literature Reviewmentioning

confidence: 99%

Behavioral Analytics for Myopic Agents

Mintz¹,

Aswani²,

Kaminsky³

et al. 2017

Preprint

View full text Add to dashboard Cite

Many multi-agent systems have the structure of a single coordinator providing behavioral or financial incentives to a large number of agents. In such settings, two challenges faced by the coordinator are a finite budget from which to allocate incentives, and an initial lack of knowledge about the utility function of the agents.Here, we present a behavioral analytics approach to solve the coordinator's problem when the agents make decisions by maximizing utility functions that depend on prior system states, inputs, and other parameters that are initially unknown and subject to partially unknown temporal dynamics. Our behavioral analytics framework involves three steps: first, we develop a behavioral model that describes the decision-making process of an agent; second, we use data to estimate behavioral model parameters for each agent and then use these estimates to predict future decisions of each agent; and third, we use the estimated behavioral model parameters to optimize a set of costly incentives to provide to each agent. In this paper, we describe a specific set of tools, models, and approaches that fit into this framework, and that adapt models and incentives as new information is collected by repeating the second and third steps of this framework. Furthermore, we prove that the incentives computed by this adaptive approach are asymptotically optimal with respect to a given loss function that describes the coordinator's objective. We optimize incentives utilizing a decomposition scheme, where each sub-problem solves the coordinator's problem for a single agent, and the master problem is a pure integer program. We conclude with a simulation study to evaluate the effectiveness of our behavioral analytics approach in designing personalized treatment plans for a weight loss program. The results show that our approach maintains efficacy of the program while reducing its costs by up to 60%, while adaptive heuristics provide substantially less savings.

show abstract

An adaptive-learning framework for semi-cooperative multi-agent coordination

Cited by 7 publications

References 67 publications

HDP Algorithms for Trajectory Tracking and Formation Control of Multi-Agent Systems

HDP Algorithms for Trajectory Tracking and Formation Control of Multi-Agent Systems

A values‐driven self‐organization mechanism for automating multiagent coordination

Behavioral Analytics for Myopic Agents

Contact Info

Product

Resources

About