Integrating recurrent neural networks and reinforcement learning for dynamic service composition

Wang, Hongbing; Li, Jiajie; Yu, Qi; Hong, Tianjing; Jia, Yan; Zhao, Wei

doi:10.1016/j.future.2020.02.030

Cited by 28 publications

(14 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, several works have been proposed to use RL methods on large-scale services composition. For instance, Wang et al [37] used hierarchical RL, Liu et al [21], [38] utilized recurrent neural networks and deep RL approaches, and Yang and Xie [45] proposed an actor-critic method as a deep RL approach.…”

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning for Interactive QoS-Aware Services Composition

Alizadeh

Osmani

Khanouche

et al. 2021

IEEE Systems Journal

View full text Add to dashboard Cite

An important and challenging research problem in web of things is how to select an appropriate composition of concrete services in a dynamic and unpredictable environment. The main goal of this article is to select from all possible compositions the optimal one without knowing a priori the users' quality of service (QoS) preferences. From a theoretical point of view, we give bounds on the problem search space. As the QoS user's preferences are unknown, we propose a vector-valued MDP approach for finding the optimal QoS-aware services composition. The algorithm alternatively solves MDP with dynamic programming and learns the preferences via direct queries to the user. An important feature of the proposed algorithm is that it is able to get the optimal composition and, at the same time, limits the number of interactions with the user. Experiments on a real-world large size dataset with more than 3500 web services show that our algorithm finds the optimal composite services with around 50 interactions with the user. Index Terms-Quality of services (QoS), reinforcement learning (RL), services composition, web of things (WoT). I. INTRODUCTION T HE web of things (WoT) opens the way for the development of new intelligent applications that can be used to provide innovative and valuable services in several domains, such as smart cities, smart homes, ambient assisted living, and connected cars [11], [35]. One of the WoT challenges is to deal with services composition to ensure suitable services flexibility and customization. This is crucial to meet the evolving needs of users, which can be expressed through several parameters of quality of service (QoS) (such as response time, throughput, availability, price, popularity, etc.). Following the web service paradigm [4] services, composition problems are specified as a workflow involving abstract and concrete services.

show abstract

Section: Related Workmentioning

confidence: 99%

Reinforcement Learning for Interactive QoS-Aware Services Composition

Alizadeh

Osmani

Khanouche

et al. 2021

IEEE Systems Journal

View full text Add to dashboard Cite

show abstract

“…This study is limited to the mobile environment, which is distinguished by limited resource storage and users that move often. Wang et al [21] proposed a service composition approach based on QoS prediction and reinforcement learning. Specifically, they used a recurrent neural network to predict the QoS.…”

Section: Related Workmentioning

confidence: 99%

Towards Dynamic Reconfiguration of a Composite Web Service: An Approach Based on QoS Prediction

et al. 2021

View full text Add to dashboard Cite

Service-oriented architecture provides the ability to combine several web services in order to fulfil a user-specific requirement. In dynamic environments, the appearance of several unforeseen events can destabilize the composite web service (CWS) and affect its quality. To deal with these issues, the composite web service must be dynamically reconfigured. Dynamic reconfiguration may be enhanced by avoiding the invocation of degraded web services by predicting QoS for the candidate web service. In this paper, we propose a dynamic reconfiguration method based on HMM (Hidden Markov Model) states to predict the imminent degradation in QoS and prevent the invocation of partner web services with degraded QoS values. PSO (Particle Swarm Optimization) and SFLA (Shuffled Frog Leaping Algorithm) are used to improve the prediction efficiency of HMM. Through extensive experiments on a real-world dataset, WS-Dream, the results demonstrate that the proposed approach can achieve better prediction accuracy. Moreover, we carried out a case study where we revealed that the proposed approach outperforms several state-of-the-art methods in terms of execution time.

show abstract

“…In [10], authors propose a solution for service composition in complex and dynamic environments based on reinforcement learning, QoS prediction and neural networks. Q-Learning algorithm (a form of reinforcement learning algorithm) is used to deal with the complex and extremely dynamic environment and QoS changing.…”

Section: Related Workmentioning

confidence: 99%

Quality-Based Reinforcement Learning in Intelligent Opportunistic Software Composition

Hacid

Trouilhet

Adreit

et al. 2021

2021 IEEE 30th International Conference on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE)

View full text Add to dashboard Cite

Internet of Things and cyber-physical systems are characterised by openness and an increasing number of devices and their associated services. In a previous work, we have proposed to exploit opportunistically these services in order to automatically make emerge customised applications that suit user preferences. For that, we have developed a generic solution for bottom-up opportunistic service composition, based on reinforcement learning. In this work, it is extended to handle more efficiently the appearance of new components using service annotation and quality attributes in order to generalise and share knowledge with new discovered services. A didactic use case is used for illustration and demonstration purposes.

show abstract

Integrating recurrent neural networks and reinforcement learning for dynamic service composition

Cited by 28 publications

References 21 publications

Reinforcement Learning for Interactive QoS-Aware Services Composition

Reinforcement Learning for Interactive QoS-Aware Services Composition

Towards Dynamic Reconfiguration of a Composite Web Service: An Approach Based on QoS Prediction

Quality-Based Reinforcement Learning in Intelligent Opportunistic Software Composition

Contact Info

Product

Resources

About