This survey paper studies deterministic control systems that integrate three of the most active research areas during the last years: (1) online learning control systems, (2) distributed control of networked multiagent systems, and (3) hybrid dynamical systems (HDSs). The interest for these types of systems has been motivated mainly by two reasons: First, the development of cheap massive computational power and advanced communication technologies, which allows to carry out large computations in complex networked systems, and second, the recent development of a comprehensive theory for HDSs that allows to integrate continuous-time dynamical systems and discrete-time dynamical systems in a unified manner, thus providing a unifying modeling language for complex learning-based control systems. In this paper, we aim to give a comprehensive survey of the current state of the art in the area of online learning control in multiagent systems, presenting an overview of the different types of problems that can be addressed, as well as the most representative control architectures found in the literature. These control architectures are modeled as HDSs, which include as special subsets continuous-time dynamical systems and discrete-time dynamical systems. We highlight the different advantages and limitations of the existing results as well as some interesting potential future directions and open problems.
KEYWORDSadaptive control, hybrid dynamical systems, learning, multiagent systems, networked systems
228One of the fields where control systems with learning and adaptation have recently seen increased attention is in the area of networked multiagent systems (MAS), 16,17 which are systems comprised of multiple interacting subsystems, each subsystem having individual dynamics and computational capabilities as well as limited sensing, communication, and actuation. Depending on the structure of the underlying communication graph between the agents of the system, MAS can be centralized or decentralized. In a centralized MAS, there exists one central agent which has access to the state information of all other agents and which is able to compute the control action for all agents of the system. On the other hand, in a decentralized or distributed MAS, agents individually compute their own control signals, which are shared with a subset of the other agents characterized by a communication graph. Figure 1 illustrates three MASs with different communication graphs. Centralized MASs are typically not scalable, and they present the disadvantage of having a single point of failure that could potentially shut down the complete system. 16 Decentralized and distributed systems, on the other hand, are scalable and more robust, and in nonstationary environments, they allow for adaptation and self-organization of the system. 18 Recent technological advances in communication and computation have made MAS ubiquitous, and today, they can be found in several engineering systems such as in power generation and distribution systems, 19-21 teams of ...