Federated Deep Reinforcement Learning-Based Spectrum Access Algorithm With Warranty Contract in Intelligent Transportation Systems

Zhu, R. Y.; Li, Mengyao; Liu, Hao; Liu, Lu

doi:10.1109/tits.2022.3179442

Cited by 4 publications

(4 citation statements)

References 48 publications

(59 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FRL is a recent topic that has been successfully adopted in several works. For example, there has been FRL application to 5G [88], autonomous driving [89], robotics [90], healthcare [91], and transportation [92]. These examples demonstrate how FRL can tackle several problems faced nowadays and foster the development of new approaches related to this combined framework.…”

Section: Federated Reinforcement Learning (Frl)mentioning

confidence: 99%

Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges

et al. 2023

View full text Add to dashboard Cite

The internet of things (IoT) represents a disruptive concept that has been changing society in several ways. There have been several successful applications of IoT in the industry. For example, in transportation systems, the novel internet of vehicles (IoV) concept has enabled new research directions and automation solutions. Moreover, reinforcement learning (RL), federated learning (FL), and federated reinforcement learning (FRL) have demonstrated remarkable success in solving complex problems in different applications. In recent years, new solutions have been developed based on this combined framework (i.e., federated reinforcement learning). Conversely, there is a lack of analysis concerning IoT applications and a standard view of challenges and future directions of the current FRL landscape. Thereupon, the main goal of this research is to present a literature review of federated reinforcement learning (FRL) applications in IoT from multiple perspectives. We focus on analyzing applications in multiple areas (e.g., security, sustainability and efficiency, vehicular solutions, and industrial services) to highlight existing solutions, their characteristics, and research gaps. Additionally, we identify key short- and long-term challenges leading to new opportunities in the field. This research intends to picture the current FRL ecosystem in IoT to foster the development of new solutions based on existing challenges.

show abstract

Section: Federated Reinforcement Learning (Frl)mentioning

confidence: 99%

Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This framework can perform collaborative training while ensuring local data privacy and greatly reducing the traffic load between the SUs and fusion center. In [47], the authors proposed a spectrum access model based on the federated deep Q network (DQN), which adopts the asynchronous federated weighted learning algorithm to share and update the weights of the DQN in multiple agents, to decrease the time cost and accelerate convergence. The decentralized nature of FL comes with its own set of challenges.…”

Section: Related Workmentioning

confidence: 99%

1-D Convolutional Neural Network-Based Models for Cooperative Spectrum Sensing

Serghini,

Semlali,

Maali

et al. 2023

Future Internet

View full text Add to dashboard Cite

Spectrum sensing is an essential function of cognitive radio technology that can enable the reuse of available radio resources by so-called secondary users without creating harmful interference with licensed users. The application of machine learning techniques to spectrum sensing has attracted considerable interest in the literature. In this contribution, we study cooperative spectrum sensing in a cognitive radio network where multiple secondary users cooperate to detect a primary user. We introduce multiple cooperative spectrum sensing schemes based on a deep neural network, which incorporate a one-dimensional convolutional neural network and a long short-term memory network. The primary objective of these schemes is to effectively learn the activity patterns of the primary user. The scenario of an imperfect transmission channel is considered for service messages to demonstrate the robustness of the proposed model. The performance of the proposed methods is evaluated with the receiver operating characteristic curve, the probability of detection for various SNR levels and the computational time. The simulation results confirm the effectiveness of the bidirectional long short-term memory-based method, surpassing the performance of the other proposed schemes and the current state-of-the-art methods in terms of detection probability, while ensuring a reasonable online detection time.

show abstract

“…It could be integrated into deep deterministic policy gradient (DDPG) [11], where it trains a set of client agents in a distributed manner, and the trained models are securely aggregated to the server agent. Federated DDPG (FDDPG) is privacy-preserving since it restricts the interaction between the local environment and agents other than the environment owner [12]. Therefore, a federated MaaS platform could be a promising solution to address the aforementioned privacy issues.…”

Section: Introductionmentioning

confidence: 99%

“…The policy obtained in Algorithm 1 will be converged, given that the critic and actor networks are updated based on Eqs. ( 9) - (12).…”

mentioning

confidence: 99%

Privacy-Preserving Federated Deep Reinforcement Learning for Mobility-as-a-Service

Chu,

Guo

2024

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Mobility-as-a-service (MaaS) is a new transport model that combines multiple transport modes in a single platform. Dynamic passenger behavior based on past experiences requires reinforcement-based optimization of MaaS services. Deep reinforcement learning (DRL) may improve passenger satisfaction by offering the most appropriate transport services based on individual passenger experiences and preferences.However, this produces a new privacy risk to the MaaS platform using the centralized DRL method. Information leakage will occur if the platform is not carefully designed with privacypreserving mechanisms. In this paper, we propose a federated deep deterministic policy gradient (FDDPG) that maximizes passenger satisfaction and MaaS long-term profit while preserving privacy. We enforce an equally weighted experience sampling mechanism to prevent sampling bias such that the solution quality of FDDPG is statistically equivalent to the centralized algorithm. During the model training and inference, information is processed locally, and only the gradients are shared, which prevents information leakage to any semi-honest participants and eavesdroppers. Secure aggregation protocol in line with the dynamic property of the mobile agent is also used in the gradient sharing step to ensure that the algorithm is prevented from inference attacks. We perform experiments on New York Citybased real-world and synthetic scenarios. The results show that the proposed FDDPG can improve the MaaS profit and passenger satisfaction by about 90% and 15%, respectively, and maintain stable training against agent dropout. Our approach and findings could enhance MaaS utility as well as facilitate passenger trust and participation in MaaS and other data-driven transportation systems.

show abstract

Federated Deep Reinforcement Learning-Based Spectrum Access Algorithm With Warranty Contract in Intelligent Transportation Systems

Cited by 4 publications

References 48 publications

Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges

Federated Reinforcement Learning in IoT: Applications, Opportunities and Open Challenges

1-D Convolutional Neural Network-Based Models for Cooperative Spectrum Sensing

Privacy-Preserving Federated Deep Reinforcement Learning for Mobility-as-a-Service

Contact Info

Product

Resources

About