Packet Delivery Maximization Using Deep Reinforcement Learning-Based Transmission Scheduling for Industrial Cognitive Radio Systems

Abstract

The performance of data aggregation in industrial wireless communications can be degraded by environmental interference on Industrial Scientific Medical (ISM) channels. In this paper, the opportunistic spectrum access capability of cognitive radio (CR) was applied to enable devices to share primary channels with the aim of enhancing the transmission performance of the WirelessHART network. We considered a linear convergecast network, where the packets generated at each device were routed to the gateway (GW) through the aid of neighboring devices. The solar-powered cognitive access points (CAPs) were deployed to improve the successful transmission probability of the packets among field devices by opportunistically allocating the primary channels to the devices for data transmissions. In this paper, we formulate the scheduling problem of long-term throughput maximization as a framework of a Markov decision process by considering the constraints of the minimum delay, the number of required ISM channels, and the harvested energy at the CAPs. Then, we propose a deep reinforcement learning-based scheduling scheme to optimally assign multiple ISM and primary channels to the field devices in each superframe to maximize the received packets at the GW. The simulation results confirmed the superiority of the proposed scheme compared to existing methods.INDEX TERMS wirelessHART, cognitive radio, markov decision process, industrial scientific medical