Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems

Leong, Alex S.; Ramaswamy, Arunselvan; Quevedo, Daniel E.; Karl, Holger; Shi, Ling

doi:10.1016/j.automatica.2019.108759

Cited by 115 publications

(103 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [116], a global routing agent with Q-Learning is proposed for weight adjustment of the state-of-the-art routing algorithm, aiming at achieving a balance between the overall delay and the lifetime of the network. The study in [117] focuses on a DRL-based sensor scheduling problem for allocating wireless channels to sensors for the purposes of remote state estimation of dynamical systems. The algorithm can be run online, and is model-free with respect to the wireless channel parameters.…”

Section: Aiot Network Layer -Iot Communication Networkmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Lei

Tan

Zheng

et al. 2020

IEEE Commun. Surv. Tutorials

210

View full text Add to dashboard Cite

The Internet of Things (IoT) extends the Internet connectivity into billions of IoT devices around the world, which collect and share information to reflect the status of physical world. The Autonomous Control System (ACS), on the other hand, performs control functions on the physical systems without external intervention over an extended period of time. The integration of IoT and ACS results in a new concept -autonomous IoT (AIoT). The sensors collect information on the system status, based on which intelligent agents in IoT devices as well as Edge/Fog/Cloud servers make control decisions for the actuators to react. In order to achieve autonomy, a promising method is for the intelligent agents to leverage the techniques in the field of artificial intelligence, especially reinforcement learning (RL) and deep reinforcement learning (DRL) for decision making. In this paper, we first provide comprehensive survey of the state-of-art research, and then propose a general model for the applications of RL/DRL in AIoT. Finally, the challenges and open issues for future research are identified.

show abstract

Section: Aiot Network Layer -Iot Communication Networkmentioning

confidence: 99%

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Lei

Tan

Zheng

et al. 2020

IEEE Commun. Surv. Tutorials

210

View full text Add to dashboard Cite

show abstract

“…C. The Existence of a Zooming Quantizer Taking (14) and (16) into (1), the plant state at t + n can be written as…”

Section: B Construction Of a Zooming Quantizermentioning

confidence: 99%

On the Latency, Rate, and Reliability Tradeoff in Wireless Networked Control Systems for IIoT

Liu

Nair

et al. 2021

IEEE Internet Things J.

View full text Add to dashboard Cite

Wireless networked control systems (WNCSs) provide a key enabling technique for Industry Internet of Things (IIoT). However, in the literature of WNCSs, most of the research focuses on the control perspective, and has considered oversimplified models of wireless communications which do not capture the key parameters of a practical wireless communication system, such as latency, data rate and reliability. In this paper, we focus on a WNCS, where a controller transmits quantized and encoded control codewords to a remote actuator through a wireless channel, and adopt a detailed model of the wireless communication system, which jointly considers the interrelated communication parameters. We derive the stability region of the WNCS. If and only if the tuple of the communication parameters lies in the region, the average cost function, i.e., a performance metric of the WNCS, is bounded. We further obtain a necessary and sufficient condition under which the stability region is nbounded, where n is the control codeword blocklength. We also analyze the average cost function of the WNCS. Such analysis is non-trivial because the finite-bit control-signal quantizer introduces a non-linear and discontinuous quantization function which makes the performance analysis very difficult. We derive tight upper and lower bounds on the average cost function in terms of latency, data rate and reliability. Our analytical results provide important insights into the design of the optimal parameters to minimize the average cost within the stability region. Index Terms-Industrial Internet of Things, sensor-actuator network, wireless networked control, mission-critical communications, performance analysis.

show abstract

“…> 0. This is the case where the term in the parenthesis in (25) is positive and we can directly apply Hoeffding's inequality (Lemma 1) to get the desired bound (19).…”

Section: Theorem 1 (Sample-based Stability Analysis)mentioning

confidence: 99%

“…In contrast our work is focused on collecting data and learning unknown channel models instead of system dynamics. In the context of networked control systems very recent works from the last two years are proposing data-based approaches including deep learning for allocating resources and scheduling [12,15,25,32,40] as well as for controller design [6,35]. A related but broader topic is also the identification of switched systems [29] where the matrix dynamics are also unknown.…”

Section: Introductionmentioning

confidence: 99%

Sample Complexity of Networked Control Systems Over Unknown Channels

Gatsis

Pappas

2018

2018 IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Recent control trends are increasingly relying on communication networks and wireless channels to close the loop for Internetof-Things applications. Traditionally these approaches are model-based, i.e., assuming a network or channel model they are focused on stability analysis and appropriate controller designs. However the availability of such wireless channel modeling is fundamentally challenging in practice as channels are typically unknown a priori and only available through data samples. In this work we aim to develop algorithms that rely on channel sample data to determine the stability and performance of networked control tasks. In this regard our work is the first to characterize the amount of channel modeling that is required to answer such a question. Specifically we examine how many channel data samples are required in order to answer with high confidence whether a given networked control system is stable or not. This analysis is based on the notion of sample complexity from the learning literature and is facilitated by concentration inequalities. Moreover we establish a direct relation between the sample complexity and the networked system stability margin, i.e., the underlying packet success rate of the channel and the spectral radius of the dynamics of the control system. This illustrates that it becomes impractical to verify stability under a large range of plant and channel configurations. We validate our theoretical results in numerical simulations.

show abstract

Deep reinforcement learning for wireless sensor scheduling in cyber–physical systems

Cited by 115 publications

References 28 publications

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

Deep Reinforcement Learning for Autonomous Internet of Things: Model, Applications and Challenges

On the Latency, Rate, and Reliability Tradeoff in Wireless Networked Control Systems for IIoT

Sample Complexity of Networked Control Systems Over Unknown Channels

Contact Info

Product

Resources

About