The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Andersen, Per‐Arne; Goodwin, Morten; Granmo, Ole‐Christoffer

doi:10.1007/978-3-030-04191-5_11

Cited by 13 publications

(14 citation statements)

References 12 publications

(21 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on this architecture, the encoded features z are used for clustering the activities. CVAEs are generative models defined in [53], which are commonly used for dimensionality reduction [54], data aug-mentation [55], and reinforcement learning [56]. Considering Doppler radar data, CVAEs have been used for synthetic data generation [57].…”

Section: ) Convolution Filter-based Methodsmentioning

confidence: 99%

Unsupervised Doppler Radar Based Activity Recognition for e-Healthcare

Karayaneva

Sharifzadeh

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Passive radio frequency (RF) sensing and monitoring of human daily activities in elderly care homes is an emerging topic. Micro-Doppler radars are an appealing solution considering their nonintrusiveness, deep penetration, and high-distance range. Unsupervised activity recognition using Doppler radar data has not received attention, in spite of its importance in case of unlabelled or poorly labelled activities in real scenarios. This study proposes two unsupervised feature extraction methods for the purpose of human activity monitoring using Doppler-streams. These include a local Discrete Cosine Transform (DCT)-based feature extraction method and a local entropy-based feature extraction method. In addition, a novel application of Convolutional Variational Autoencoder (CVAE) feature extraction is employed for the first time for Doppler radar data. The three feature extraction architectures are compared with the previously used Convolutional Autoencoder (CAE) and linear feature extraction based on Principal Component Analysis (PCA) and 2DPCA. Unsupervised clustering is performed using K-Means and K-Medoids. The results show the superiority of DCT-based method, entropy-based method, and CVAE features compared to CAE, PCA, and 2DPCA, with more than 5%-20% average accuracy. In regards to computation time, the two proposed methods are noticeably much faster than the existing CVAE. Furthermore, for high-dimensional data visualisation, three manifold learning techniques are considered. The methods are compared for the projection of raw data as well as the encoded CVAE features. All three methods show an improved visualisation ability when applied to the encoded CVAE features.

show abstract

Section: ) Convolution Filter-based Methodsmentioning

confidence: 99%

Unsupervised Doppler Radar Based Activity Recognition for e-Healthcare

Karayaneva

Sharifzadeh

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In [6], Ha et al proposed World Model, an architecture for modeling the environment using a VAE model and a recurrent neural network (RNN) model, which shows that the agent can learn the optimal policy only use generate training samples. Similarly, Anderson et al [1] proposed Dreaming Variational Autoencoder, an architecture for modeling the environment using VAE and RNN, which uses the real trajectories from the actual environment to imitate the behavior of the actual environment. Conversely, Anderson et al [2] found that in high-dimensional tasks, simple heuristics exploration are often trapped in local minima of the state space, which may cause the generative model to become inaccurate or even collapse.…”

Section: Related Workmentioning

confidence: 99%

“…However, in many scenarios, the training sample may be difficult or time-consuming to obtain. Thus, some researchers attempt to represent the actual environment by using a generative model [1,6,8] to improve sample efficiency. When the generative model is sufficiently trained, the DRL algorithm can be trained without interacts with the actual environment.…”

Section: Introductionmentioning

confidence: 99%

“…When the generative model is sufficiently trained, the DRL algorithm can be trained without interacts with the actual environment. In [1,2,6,8], it is confirmed that the agent can learn the optimal policy only use generate training samples. However, these generative models may become inaccurate and even collapse where the state-action pair insufficient explored [1,2,6].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Curiosity-Driven Variational Autoencoder for Deep Q Network

Han

Zhang

Wang

et al. 2020

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

In recent years, deep reinforcement learning (DRL) has achieved tremendous success in high-dimensional and large-scale space control and sequential decision-making tasks. However, the current model-free DRL methods suffer from low sample efficiency, which is a bottleneck that limits their performance. To alleviate this problem, some researchers used the generative model for modeling the environment. But the generative model may become inaccurate or even collapse if the state has not been sufficiently explored. In this paper, we introduce a model called Curiosity-driven Variational Autoencoder (CVAE), which combines variational autoencoder and curiosity-driven exploration. During the training process, the CVAE model can improve sample efficiency while curiosity-driven exploration can make sufficient exploration in a complex environment. Then, a CVAE-based algorithm is proposed, namely DQN-CVAE, that scales CVAE to higher dimensional environments. Finally, the performance of our algorithm is evaluated through several Atari 2600 games, and the experimental results show that the DQN-CVAE achieves better performance in terms of average reward per episode on these games.

show abstract

“…Deep learning has been used for a myriad of applications ranging from games to medicine, but its applicability has only partly been explored for fish classification [10][11][12][13][14]. A specific Convolutional Neural Network (CNN) called Fast R-CNN has been applied for object detection to extract the fish from images taken in natural environment and actively ignoring background noise [6].…”

Section: Introductionmentioning

confidence: 99%

Temperate fish detection and classification: a deep learning based approach

Wiklund²,

et al. 2021

Self Cite

View full text Add to dashboard Cite

A wide range of applications in marine ecology extensively uses underwater cameras. Still, to efficiently process the vast amount of data generated, we need to develop tools that can automatically detect and recognize species captured on film. Classifying fish species from videos and images in natural environments can be challenging because of noise and variation in illumination and the surrounding habitat. In this paper, we propose a two-step deep learning approach for the detection and classification of temperate fishes without pre-filtering. The first step is to detect each single fish in an image, independent of species and sex. For this purpose, we employ the You Only Look Once (YOLO) object detection technique. In the second step, we adopt a Convolutional Neural Network (CNN) with the Squeeze-and-Excitation (SE) architecture for classifying each fish in the image without pre-filtering. We apply transfer learning to overcome the limited training samples of temperate fishes and to improve the accuracy of the classification. This is done by training the object detection model with ImageNet and the fish classifier via a public dataset (Fish4Knowledge), whereupon both the object detection and classifier are updated with temperate fishes of interest. The weights obtained from pre-training are applied to post-training as a priori. Our solution achieves the state-of-the-art accuracy of 99.27% using the pre-training model. The accuracies using the post-training model are also high; 83.68% and 87.74% with and without image augmentation, respectively. This strongly indicates that the solution is viable with a more extensive dataset.

show abstract

The Dreaming Variational Autoencoder for Reinforcement Learning Environments

Cited by 13 publications

References 12 publications

Unsupervised Doppler Radar Based Activity Recognition for e-Healthcare

Unsupervised Doppler Radar Based Activity Recognition for e-Healthcare

Curiosity-Driven Variational Autoencoder for Deep Q Network

Temperate fish detection and classification: a deep learning based approach

Contact Info

Product

Resources

About