Decentralized Control of Adaptive Data Collection Processes Based on Equilibrium Concept and Reinforcement Learning

Abstract

The model of decentralized control of adaptive data collection processes has been developed based on the equilibrium concept, which is used to study the problem of coordinating joint collective actions from the point of view of finding an effective scheme for complementing the individual actions of data collection processes in the absence of a control center. The method of decentralized control of adaptive data collection processes in autonomous distributed systems based on the equilibrium concept and reinforcement learning by the method of normalized exponential function (softmax) has been developed. The method allows one to organize autonomous distributed exploration under the conditions of dynamic changes in the number of data collection processes and unreliable local information interaction between them. As a result of research and modeling of the developed method of decentralized control, it has been established that the use of the reinforcement learning (normalized exponential function method) provides more effective search for a solution compared to the method of adaptive random search (by an average of 28.3%). Using the efficiency retention rate, an estimate was obtained for the dependence of the work of the developed decentralized control method on the change in the number of adaptive data collection processes and the change in the information interaction channels between adaptive data collection processes.

Adaptive Learning Service Based on Spacing Effect

“…The environment is usually presented as a Markov decision-making process (MDP), as many learning algorithms use dynamic programming techniques to reinforce this context. The main difference between classical active programming methods and supported learning algorithms is that the latter does not require knowledge of the exact mathematical model of MDP, and they target large MDPs where accurate methods become impossible [6].…”

“…Decentralization [6] is the foundation that gives companies more independence over their collaboration, data ownership, better privacy and security, a more robust and resilient communications infrastructure, and massive scalability.…”