Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey

Abdellatif, Alaa Awad; Mhaisen, Naram; Chkirbene, Zina; Mohamed, Amr; Erbad, Aiman; Guizani, Mohsen

doi:10.48550/arxiv.2108.04087

Cited by 5 publications

(3 citation statements)

References 143 publications

(191 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since network demand constantly changes, we opt to use a dynamic optimization approach. Because of its ability to adapt to different service requirements and manage highly complex environments, with low complexity [47], [48], Deep Reinforcement Learning (DRL) was used as the primary method of optimizing network slices. In our case, DRL was used to ensure the optimal choice of paths and resources while the state of the network might be unstable (e.g., When some network links fail).…”

Section: B Network Slice Creation and Optimization (Phase 2)mentioning

confidence: 99%

ECP: Error-Aware, Cost-Effective and Proactive Network Slicing Framework

Aboeleneen,

Abdellatif,

Erbad

et al. 2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

Recent advancements in Software Defined Networks (SDN), Open Radio Access Network (O-RAN), and 5G technology have significantly expanded the capabilities of wireless networks, extending beyond mere data transmission. This progression has led to the emergence of Virtual Networks (VN) and Network Slicing, enabling industries to enhance their services and applications by establishing virtual networks that utilize shared physical infrastructure. Many works in the literature have considered optimizing the allocation of on-demand slices, assuming the absolute availability of resources and their accurate load. However, accurately allocating future network slices remains challenging due to the error in load prediction, diverse Key Performance Indicators (KPIs), resource price variations, and the potential for overor under-provisioning. This study presents a two-phase intelligent approach to address these challenges. The framework proactively predicts different slice loads while considering prediction errors in optimizing future slices with varied KPIs in a cost-efficient manner. Specifically, our method utilizes historical load data per service and employs AI-based forecasts for service load prediction. Subsequently, it employs a Deep Reinforcement Learning (DRL) agent on O-RAN's virtual Control Unit (vCU) and virtual Distributed unit (vDU) to correct errors in prediction and optimize the cost of slice allocation based on service KPI requirements, ultimately pre-allocating future network slices at reduced costs. Through experimental validation against various baselines and state-of-the-art solutions, we demonstrate the efficacy of our proposed solution, achieving a notable reduction (37-51%) in the average cost of allocated slices while inquiring about (1.5-7%

show abstract

Section: B Network Slice Creation and Optimization (Phase 2)mentioning

confidence: 99%

ECP: Error-Aware, Cost-Effective and Proactive Network Slicing Framework

Aboeleneen,

Abdellatif,

Erbad

et al. 2024

IEEE Open J. Commun. Soc.

View full text Add to dashboard Cite

show abstract

“…Furthermore, machine learning (ML) methods have been applied in the healthcare industry to provide smart health services, optimizing system parameters, monitoring populations, and controlling chronic diseases [6]. In particular, the reinforcement learning (RL) and deep RL (DRL) methods are used in the healthcare industry to maximize energy efficiency, minimize communication latency, and allocate efficient resources [7] [8]. Federated learning (FL) is a recent ML paradigm that allows heterogeneous edge nodes to train data models and perform aggregation centrally, protecting data privacy.…”

Section: Introductionmentioning

confidence: 99%

FDRL Approach for Association and Resource Allocation in Multi-UAV Air-To-Ground IoMT Network

Erbad¹,

Abishu²,

Albaseer³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

<p>In 6G networks, unmanned aerial vehicles (UAVs) can serve as aerial flying base stations (AFBS) with aerial mobile edge computing (AMEC) server capabilities. AFBS is an increasingly popular solution for delivering time-sensitive applications, extending network coverage, and assisting ground base stations in the healthcare systems for remote areas with limited infrastructure. Furthermore, the UAVs are deployed in the healthcare system to support the Internet of medical things (IoMT) devices in data collection, medical equipment distribution, and providing smart services. However, ensuring the privacy and security of patients' data with the limited UAV resources is a major challenge. In this paper, we present a federated deep reinforcement learning framework for resource allocation in UAV-enabled healthcare systems, where IoMT devices send their trained model parameters without transmitting sensitive raw data to the AMEC server. In the proposed framework, the IoMT device is associated with AFBS based on the quality of the data and its demand in order to maximize learning efficiency and accuracy. This work aims to minimize the computation costs of the IoMT devices while considering UAV resources and the fairness of UAV coverage. Simulation results prove that our proposed algorithm outperforms other baseline algorithms in learning accuracy and computational cost.</p>

show abstract

“…Among those approaches, RL has become an attractive approach for constructing optimal dynamic treatment regimes in healthcare to monitor chronic disease [12]. A multimodal RL algorithm is used to maximize the battery life of IoT devices through data compression, energy efficient communication, and minimizing latency in medical IoT systems, particularly for emergency cases [13].…”

mentioning

confidence: 99%

Multiagent Federated Reinforcement Learning for Resource Allocation in UAV-Enabled Internet of Medical Things Networks

Seid

Erbad

Abishu

et al. 2023

IEEE Internet Things J.

View full text Add to dashboard Cite

In the 5G/B5G network paradigms, intelligent medical devices known as the Internet of Medical Things (IoMT) have been used in the healthcare industry to monitor remote users' health status, such as elderly monitoring, injuries, stress, and patients with chronic diseases. Since IoMT devices have limited resources, mobile edge computing (MEC) has been deployed in 5G networks to enable them to offload their tasks to the nearest computational servers for processing. However, when IoMTs are far from network coverage or the computational servers at the terrestrial MEC are overloaded/emergencies occur, these devices cannot access computing services, potentially risking the lives of patients. In this context, unmanned aerial vehicles (UAVs) are considered a prominent aerial connectivity solution for healthcare systems. In this paper, we propose a multiagent federated reinforcement learning (MAFRL)-based resource allocation framework for a multi-UAV-enabled healthcare system. We formulate the computation offloading and resource allocation problems as a Markov decision process game in federated learning with multiple participants. Then, we propose a MAFRL algorithm to solve the formulated problem, minimize latency and energy consumption, and ensure the quality of service. Finally, extensive simulation results on a real-world heartbeat dataset prove that the proposed MAFRL algorithm significantly minimizes the cost, preserves privacy, and improves accuracy compared to the baseline learning algorithms.

show abstract

Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey

Cited by 5 publications

References 143 publications

ECP: Error-Aware, Cost-Effective and Proactive Network Slicing Framework

ECP: Error-Aware, Cost-Effective and Proactive Network Slicing Framework

FDRL Approach for Association and Resource Allocation in Multi-UAV Air-To-Ground IoMT Network

Multiagent Federated Reinforcement Learning for Resource Allocation in UAV-Enabled Internet of Medical Things Networks

Contact Info

Product

Resources

About