Integrating Reinforcement Learning with Multi-Agent Techniques for Adaptive Service Composition

Abstract

Service-oriented architecture is a widely used software engineering paradigm to cope with complexity and dynamics in enterprise applications. Service composition, which provides a cost-effective way to implement software systems, has attracted significant attention from both industry and research communities. As online services may keep evolving over time and thus lead to a highly dynamic environment, service composition must be self-adaptive to tackle uninformed behavior during the evolution of services. In addition, service composition should also maintain high efficiency for large-scale services, which are common for enterprise applications. This article presents a new model for large-scale adaptive service composition based on multi-agent reinforcement learning. The model integrates reinforcement learning and game theory, where the former is to achieve adaptation in a highly dynamic environment and the latter is to enable agents to work for a common task (i.e., composition). In particular, we propose a multi-agent Q-learning algorithm for service composition, which is expected to achieve better performance when compared with the single-agent Q-learning method and multi-agent SARSA (State-Action-Reward-State-Action) method. Our experimental results demonstrate the effectiveness and efficiency of our approach.

RETRACTED: Research on decision-making strategy of soccer robot based on multi-agent reinforcement learning

“…Multi-agent system (a distributed system composed of multiple independent autonomous agents, which are in the same working environment, can sense the environmental information and perform their own actions) robot soccer match is a typical multi-agent system research platform, which is also a field of artificial intelligence and robotics machine learning. 1,2 At present, the subject of high challenge has received extensive attention and research. 3,4,5 However, the process of robot soccer match is complex, dynamic and uncertain, which makes the decision-making system based on expert knowledge is lack sufficient completeness and flexibility to deal the process of the complex, dynamic and uncertain of robot football game, however, the reinforcement learning method does not need accurate environment model and complete expert knowledge.…”

Sequential Learning-based IaaS Composition

“…First, we analyze the efficiency of the proposed long-term qualitative composition framework for a new set of requests (without history). We compare the proposed approaches with four state-of-the-art techniques: a) Global Dynamic Programming [77], b) 2-d Q-learning [43], c) Heuristics based optimization [40], and d) On-policy SARSA learning approach [61].…”

“…Finally, long-term requests are accepted through collaborative decisions of local optimizations, i.e., based on the global utility score. • On-policy SARSA learning approach: A modified SARSA (State-Action-Reward-State-Action) algorithm is proposed as the on-policy reinforcement learning approach for adaptive service composition [61]. The difference between SARSA and Q-learning is that SARSA selects requests (action) following the same current policy and updates its Q-values.…”

Self-improving system integration: Mastering continuous change

“…Last but not least, the work of Wang et al [111] has been identified to fulfil the above-mentioned criteria. In their work, reinforcement learning techniques are combined with multiagent technology in the context of self-adaptive service composition.…”