A deep reinforcement learning-based algorithm for reliability-aware multi-domain service deployment in smart ecosystems

Abstract

The transition towards full network virtualization will see services for smart ecosystems including smart metering, healthcare and transportation among others, being deployed as Service Function Chains (SFCs) comprised of an ordered set of virtual network functions. However, since such services are usually deployed in remote cloud networks, the SFCs may transcend multiple domains belonging to different Infrastructure Providers (InPs), possibly with differing policies regarding billing and Quality-of-service (QoS) guarantees. Therefore, efficiently allocating the exhaustible network resources to the different SFCs while meeting the stringent requirements of the services such as delay and QoS among others, remains a complex challenge, especially under limited information disclosure by the InPs. In this work, we formulate the SFC deployment problem across multiple domains focusing on delay constraints, and propose a framework for SFC orchestration which adheres to the privacy requirements of the InPs. Then, we propose a Reinforcement Learning (RL) basedalgorithm for partitioning the SFC request across the different InPs while considering service reliability across the participating InPs. Such RL based algorithms have the intelligence to infer undisclosed InP information from historical data obtained from past experiences. Simulation results, considering both online and offline scenarios, reveal that the proposed algorithm results in

Combining simulation models and machine learning in healthcare management: strategies and applications

“…In particular, the examination of the full-text was focused on distinguishing those articles actually combining simulation approaches with ML ones and thereby excluding those works referring to pure ML algorithms, i.e. focused on the use of ML algorithms as the only modeling strategy [50][51][52] but also ascertaining the real actuation of the research in the helthcare field [53][54][55]. The full-text reading also helped in identifying: (i) additional duplicates; (ii) items not meeting the eligibility criteria that passed the previous selection step; (iii) studies not providing accurate and exhaustive descriptions of the ML and simulation approaches adopted.…”

Survivable SFC deployment method based on federated learning in multi-domain networks

“…However, this method is difficult to solve in the face of dynamically changing network resources and SFC demand scenarios, and does not consider the isolation requirements between domains. Kibalya et al proposed a multi-domain SFC deployment framework for time delay constraints, and designed a cross domain SFC partition and scheduling algorithm based on reinforcement learning method, which guaranteed the SFC deployment success rate and low mapping optimization cost in online and offline scenarios respectively [17]. However, this method also ignores the isolation requirements between domains, and does not consider the reliability index of SFC.…”

“…Specifically, ALL-MSDB algorithm backs up all VNF requests, NO-MSDB algorithm only performs deployment processing for all VNF requests without backup, RND-MSDB algorithm determines the backup scheme of each VNF request randomly. In addition, in order to measure the overall effect of the algorithms proposed in this paper, three relatively advanced algorithms are selected as the benchmark algorithms: the multi-domain SDN network service deployment algorithm named COMP1 proposed in literature [11], the reliable multi-domain SFC deployment algorithm named COMP2 based on deep reinforcement learning proposed in literature [17], and the distributed SFC multi-stage mapping algorithm named COMP3 proposed in literature [12].…”

“…In view of these shortcomings, some studies have solved the large-scale SFC deployment problem in dynamic scenarios by using relevant machine learning methods [14][15]. In addition, some studies have proposed effective SFC deployment algorithms for the isolation requirements between network domains [16][17]. Some authors have conducted deployment method research for the reliability and security requirements of SFC in multi-domain networks [18][19].…”

Contribution to multi-domain network slicing : resource orchestration framework and algorithms

“…Machine Learning (ML) has emerged as a promising technique in the domain of network management due to its ability to make intelligent decisions in dynamic and fuzzy environments. The works in [28,[79][80][81][82], adopted ML techniques to the problem of SFC provisioning in single domain networks.…”