Variational Autoencoder

Cinelli, Lucas Pinheiro; Marins, Matheus Araújo; Silva, Eduardo Antúnio Barros da; Netto, Sérgio L.

doi:10.1007/978-3-030-70679-1_5

Cited by 27 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[ 12 , 27 ]. A Variational Autoencoder (VAE) assumes that the data are sampled from an arbitrary statistical distribution [ 28 ]. It is trained in an unsupervised manner with an encoder that provides a low-dimensional latent representation of the data vector, and a decoder which attempts to reconstruct the input vector.…”

Section: Resultsmentioning

confidence: 99%

“…It is trained in an unsupervised manner with an encoder that provides a low-dimensional latent representation of the data vector, and a decoder which attempts to reconstruct the input vector. The encoder transforms its input into the parameters of a multidimensional statistical distribution, and sampling occurs where a point is drawn from the encoded distribution and fed into the decoder [ 28 ]. It can be seen as a probabilistic version of AE that can generate new data and transform existing data within an encoding–modification–decoding scheme [ 29 ].…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

A Systematic Review of Deep Learning Methodologies Used in the Drug Discovery Process with Emphasis on In Vivo Validation

Koutroumpa

Papavasileiou

Papadiamantis

et al. 2023

IJMS

View full text Add to dashboard Cite

The discovery and development of new drugs are extremely long and costly processes. Recent progress in artificial intelligence has made a positive impact on the drug development pipeline. Numerous challenges have been addressed with the growing exploitation of drug-related data and the advancement of deep learning technology. Several model frameworks have been proposed to enhance the performance of deep learning algorithms in molecular design. However, only a few have had an immediate impact on drug development since computational results may not be confirmed experimentally. This systematic review aims to summarize the different deep learning architectures used in the drug discovery process and are validated with further in vivo experiments. For each presented study, the proposed molecule or peptide that has been generated or identified by the deep learning model has been biologically evaluated in animal models. These state-of-the-art studies highlight that even if artificial intelligence in drug discovery is still in its infancy, it has great potential to accelerate the drug discovery cycle, reduce the required costs, and contribute to the integration of the 3R (Replacement, Reduction, Refinement) principles. Out of all the reviewed scientific articles, seven algorithms were identified: recurrent neural networks, specifically, long short-term memory (LSTM-RNNs), Autoencoders (AEs) and their Wasserstein Autoencoders (WAEs) and Variational Autoencoders (VAEs) variants; Convolutional Neural Networks (CNNs); Direct Message Passing Neural Networks (D-MPNNs); and Multitask Deep Neural Networks (MTDNNs). LSTM-RNNs were the most used architectures with molecules or peptide sequences as inputs.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

A Systematic Review of Deep Learning Methodologies Used in the Drug Discovery Process with Emphasis on In Vivo Validation

Koutroumpa

Papavasileiou

Papadiamantis

et al. 2023

IJMS

View full text Add to dashboard Cite

show abstract

“…The high expressiveness of the Gaussian distribution allows it to describe many phenomena in the real world. According to [30], the Gaussian distribution assumption allows VAE to utilize the reparameterization trick, which enhances training efficiency without reducing its fitting capability. VAE first approximates the Gaussian distribution 𝑁 (𝜇, 𝜎 2 ) by computing 𝜇 and 𝜎 with two neural network models.…”

Section: Vae Encodermentioning

confidence: 99%

“…We introduce non-linearity into the model by adding an activation layer. Following the common practice in [30], we choose ReLU and sigmoid as the activation function for 𝜇 and 𝜎, respectively (Equations 8 and 9).…”

Section: Vae Encodermentioning

confidence: 99%

KDDT: Knowledge Distillation-Empowered Digital Twin for Anomaly Detection

Xu,

Ali,

Yue

et al. 2023

Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Enginee

View full text Add to dashboard Cite

Cyber-physical systems (CPSs), like train control and management systems (TCMS), are becoming ubiquitous in critical infrastructures. As safety-critical systems, ensuring their dependability during operation is crucial. Digital twins (DTs) have been increasingly studied for this purpose owing to their capability of runtime monitoring and warning, prediction and detection of anomalies, etc. However, constructing a DT for anomaly detection in TCMS necessitates sufficient training data and extracting both chronological and context features with high quality. Hence, in this paper, we propose a novel method named KDDT for TCMS anomaly detection. KDDT harnesses a language model (LM) and a long short-term memory (LSTM) network to extract contexts and chronological features, respectively. To enrich data volume, KDDT benefits from out-ofdomain data with knowledge distillation (KD). We evaluated KDDT with two datasets from our industry partner Alstom and obtained the F1 scores of 0.931 and 0.915, respectively, demonstrating the effectiveness of KDDT. We also explored individual contributions of the DT model, LM, and KD to the overall performance of KDDT, via a comprehensive empirical study, and observed average F1 score improvements of 12.4%, 3%, and 6.05%, respectively. CCS CONCEPTS• Software and its engineering → Maintaining software; • Security and privacy → Intrusion detection systems; • Computer systems organization → Embedded systems; Sensors and actuators; Embedded and cyber-physical systems; • Computing methodologies → Neural networks.

show abstract

“…During the model training process, which is usually conducted based on an Expectation-Maximization meta-algorithm, the encoding distribution was “regularized”, so that the resulting latent space sufficed to generate new and meaningful datasets. The detailed mathematical derivation will be discussed in Section 3 , and readers can also refer to [ 34 ] for more technical details. The VAE model was first proposed by Kingma and Welling [ 35 ], and has been widely applied in different disciplines, for example, image generation, data classification and dimensionality reduction [ 36 , 37 , 38 ].…”

Section: Introductionmentioning

confidence: 99%

Application of Variational AutoEncoder (VAE) Model and Image Processing Approaches in Game Design

Mak

Han

Yin

2023

Sensors

View full text Add to dashboard Cite

In recent decades, the Variational AutoEncoder (VAE) model has shown good potential and capability in image generation and dimensionality reduction. The combination of VAE and various machine learning frameworks has also worked effectively in different daily life applications, however its possible use and effectiveness in modern game design has seldom been explored nor assessed. The use of its feature extractor for data clustering has also been minimally discussed in the literature neither. This study first attempts to explore different mathematical properties of the VAE model, in particular, the theoretical framework of the encoding and decoding processes, the possible achievable lower bound and loss functions of different applications; then applies the established VAE model to generate new game levels based on two well-known game settings; and to validate the effectiveness of its data clustering mechanism with the aid of the Modified National Institute of Standards and Technology (MNIST) database. Respective statistical metrics and assessments are also utilized to evaluate the performance of the proposed VAE model in aforementioned case studies. Based on the statistical and graphical results, several potential deficiencies, for example, difficulties in handling high-dimensional and vast datasets, as well as insufficient clarity of outputs are discussed; then measures of future enhancement, such as tokenization and the combination of VAE and GAN models, are also outlined. Hopefully, this can ultimately maximize the strengths and advantages of VAE for future game design tasks and relevant industrial missions.

show abstract

Variational Autoencoder

Cited by 27 publications

References 11 publications

A Systematic Review of Deep Learning Methodologies Used in the Drug Discovery Process with Emphasis on In Vivo Validation

A Systematic Review of Deep Learning Methodologies Used in the Drug Discovery Process with Emphasis on In Vivo Validation

KDDT: Knowledge Distillation-Empowered Digital Twin for Anomaly Detection

Application of Variational AutoEncoder (VAE) Model and Image Processing Approaches in Game Design

Contact Info

Product

Resources

About