Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to Atari Breakout game

Patel, Devdhar; Hazan, Hananel; Saunders, Daniel J.; Siegelmann, Hava T.; Kozma, Róbert

doi:10.1016/j.neunet.2019.08.009

Cited by 65 publications

(30 citation statements)

References 42 publications

(60 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous study [15] shows that SNN could improve robustness to occlusion in the input image. To verify the robustness of DSQN, DSQN and DQN are evaluated under the whitebox attacks respectively.…”

Section: Robustness To White-box Attacksmentioning

confidence: 95%

Deep Reinforcement Learning with Spiking Q-learning

Chen¹,

Peng²,

Huang³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the help of special neuromorphic hardware, spiking neural networks (SNNs) are expected to realize artificial intelligence with less energy consumption. It provides a promising energy-efficient way for realistic control tasks by combing SNNs and deep reinforcement learning (RL). There are only a few existing SNN-based RL methods at present. Most of them either lack generalization ability or employ Artificial Neural Networks (ANNs) to estimate value function in training. The former needs to tune numerous hyper-parameters for each scenario, and the latter limits the application of different types of RL algorithm and ignores the large energy consumption in training. To develop a robust spike-based RL method, we draw inspiration from non-spiking interneurons found in insects and propose the deep spiking Q-network (DSQN), using the membrane voltage of nonspiking neurons as the representation of Q-value, which can directly learn robust policies from highdimensional sensory inputs using end-to-end RL. Experiments conducted on 17 Atari games demonstrate the effectiveness of DSQN by outperforming the ANN-based deep Q-network (DQN) in most games. Moreover, the experimental results show superior learning stability and robustness to adversarial attacks of DSQN.

show abstract

Section: Robustness To White-box Attacksmentioning

confidence: 95%

Deep Reinforcement Learning with Spiking Q-learning

Chen¹,

Peng²,

Huang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Paper [8], shows comparisons between RELU Neural Network and Spiking Neural Network in terms of rewards achieved for given number of epochs in breakout, using Epsilon greedy approach and conventional greedy approach. Furthermore, with additional benefits of SNNs can supplement the working of DQN when data is noisy and incomplete [8].…”

Section: Related Research Workmentioning

confidence: 99%

The Use of Reinforcement Learning in Gaming The Breakout Game Case Study.pdf

Chen¹,

Dewan²,

Trivedi³

et al. 2020

Preprint

View full text Add to dashboard Cite

This paper provides a comparative analysis between Deep Q Network (DQN) and Double Deep Q Network (DDQN) algorithms based on their hit rate, out of which DDQN proved to be better for Breakout game. DQN is chosen over Basic Q learning because it understands policy learning using its neural network which is good for complex environment and DDQN is chosen as it solves overestimation problem (agent always choses non-optimal action for any state just because it has maximum Q-value) occurring in basic Q-learning.

show abstract

“…Such a modelfree approach could be Q-learning, DQN, and so on. As the state-of-art Liquid State Machine (LSM) is believed as a great improvement over the theory of artificial neural networks, it is suggested to implement DQN with LSM for complex and large-scale networks [43]. In this paper, we choose Q-learning regarding its simplicity and the good interpretability to our task dependency scenario.…”

Section: E Problem Formulationmentioning

confidence: 99%

Dependency-Aware Computation Offloading in Mobile Edge Computing: A Reinforcement Learning Approach

Pan

Zhi-yong²,

Zhang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Mobile edge computing (MobEC) builds an Information Technology (IT) service environment to enable cloud-computing capabilities at the edge of mobile networks. To tackle the restrictions in the battery power and computation capability of mobile devices, task offloading for using MobEC is developed and used to reduce the service latency and to ensure high service efficiency. However, most of the existing schemes only focus on one-shot offloading, while taking less into consideration the task dependency. It is urgently needed a more comprehensive and adaptive way to take both the energy constraint and the inherent dependency of tasks into account, since modern communication networks have increasingly become complicated and dynamic. To this end, in this paper, we are motivated to study the problem of dependencyaware task offloading decision in MobEC, aiming at minimizing the execution time for mobile applications with constraints on energy consumption. To solve this problem, we propose a model-free approach based on reinforcement learning (RL), i.e., a Q-learning approach that adaptively learns to optimize the offloading decision and energy consumption jointly by interacting with the network environment. Simulation results show that our RL-based approach is able to achieve significant reduction on the total execution time with comparably less energy consumption.INDEX TERMS Mobile edge computing, offloading, resource allocation, reinforcement learning, task dependency.

show abstract

Improved robustness of reinforcement learning policies upon conversion to spiking neuronal network platforms applied to Atari Breakout game

Cited by 65 publications

References 42 publications

Deep Reinforcement Learning with Spiking Q-learning

Deep Reinforcement Learning with Spiking Q-learning

The Use of Reinforcement Learning in Gaming The Breakout Game Case Study.pdf

Dependency-Aware Computation Offloading in Mobile Edge Computing: A Reinforcement Learning Approach

Contact Info

Product

Resources

About