Improving Adversarial Text Generation by Modeling the Distant Future

Zhang, Ruiyi; Chen, Changyou; Gan, Zhe; Wang, Wenlin; Shen, Dinghan; Wang, Guoyin; Wen, Zheng; Carin, Lawrence

doi:10.18653/v1/2020.acl-main.227

Cited by 26 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Based on GAN, many algorithms have been developed, such as conditional GAN (Mirza & Osindero, ), StackGAN (Zhang et al, ), and GP‐GAN (Wu, Zheng, Zhang, & Huang, ). In particular, adversarial training has been used for generating realistic text (Zhang et al, ).…”

Section: Deep Learningmentioning

confidence: 99%

From shallow feature learning to deep learning: Benefits from the width and depth of deep architectures

Zhong

Ling

Wang

2018

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Since Pearson developed principal component analysis (PCA) in 1901, feature learning (or called representation learning) has been studied for more than 100 years. During this period, many “shallow” feature learning methods have been proposed based on various learning criteria and techniques, until the popular deep learning research in recent years. In this advanced review, we describe the historical profile of the shallow feature learning research and introduce the important developments of the deep learning models. Particularly, we survey the deep architectures with benefits from the optimization of their width and depth, as these models have achieved new records in many applications, such as image classification and object detection. Finally, several interesting directions of deep learning are presented and briefly discussed. This article is categorized under: Technologies > Classification Technologies > Machine Learning Technologies > Computational Intelligence

show abstract

Section: Deep Learningmentioning

confidence: 99%

From shallow feature learning to deep learning: Benefits from the width and depth of deep architectures

Zhong

Ling

Wang

2018

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…The model is tested on four different datasets including Yelp, Amazon, Film, Obama Speech and the reported results showed that it outperforms all three baseline methods used for comparison (Markov Chain, RNN and Seq2Seq). A similar text generation approach using adversarial networks is used in [12] where a special focus is placed on the generative process over a longer horizon in order the model be able to capture the semantic meaning in long text generation.…”

Section: Related Workmentioning

confidence: 99%

Towards Improved Classification Accuracy on Highly Imbalanced Text Dataset Using Deep Neural Language Models

et al. 2021

View full text Add to dashboard Cite

Data imbalance is a frequently occurring problem in classification tasks where the number of samples in one category exceeds the amount in others. Quite often, the minority class data is of great importance representing concepts of interest and is often challenging to obtain in real-life scenarios and applications. Imagine a customers’ dataset for bank loans-majority of the instances belong to non-defaulter class, only a small number of customers would be labeled as defaulters, however, the performance accuracy is more important on defaulters labels than non-defaulter in such highly imbalance datasets. Lack of enough data samples across all the class labels results in data imbalance causing poor classification performance while training the model. Synthetic data generation and oversampling techniques such as SMOTE, AdaSyn can address this issue for statistical data, yet such methods suffer from overfitting and substantial noise. While such techniques have proved useful for synthetic numerical and image data generation using GANs, the effectiveness of approaches proposed for textual data, which can retain grammatical structure, context, and semantic information, has yet to be evaluated. In this paper, we address this issue by assessing text sequence generation algorithms coupled with grammatical validation on domain-specific highly imbalanced datasets for text classification. We exploit recently proposed GPT-2 and LSTM-based text generation models to introduce balance in highly imbalanced text datasets. The experiments presented in this paper on three highly imbalanced datasets from different domains show that the performance of same deep neural network models improve up to 17% when datasets are balanced using generated text.

show abstract

“…To tackle this issue of non-differentiability, policy learning has been suggested, which suffers from high variance during training (13,67). Therefore, specialized distributions, such as the Gumbel-softmax (68, 69), a concrete distribution (70), or a soft-argmax function (71), have been proposed to approximate the gradient of the model from discrete samples.…”

Section: A2 Conditional Generation With Autoencodersmentioning

confidence: 99%

Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics

Das,

Sercu,

Wadhawan

et al. 2020

Preprint

View full text Add to dashboard Cite

De novo therapeutic design is challenged by a vast chemical repertoire and multiple constraints such as high broad-spectrum potency and low toxicity. We propose CLaSS (Controlled Latent attribute Space Sampling) a novel and efficient computational method for attribute-controlled generation of molecules, which leverages guidance from classifiers trained on an informative latent space of molecules modeled using a deep generative autoencoder. We further screen the generated molecules by using a set of deep learning classifiers in conjunction with novel physicochemical features derived from highthroughput molecular simulations. The proposed approach is employed for designing non-toxic antimicrobial peptides (AMPs) with strong broad-spectrum potency, which are emerging drug candidates for tackling antibiotic resistance.Synthesis and wet lab testing of only twenty designed sequences identified two novel and minimalist AMPs with high potency against diverse Gram-positive and Gram-negative pathogens, including the hard-to-treat multidrug-resistant K. pneumoniae, as well as low in vitro and in vivo toxicity. The proposed approach thus presents a viable path for faster discovery of potent and selective broad-spectrum antimicrobials with a higher success rate than state-of-the-art methods.

show abstract

Improving Adversarial Text Generation by Modeling the Distant Future

Cited by 26 publications

References 22 publications

From shallow feature learning to deep learning: Benefits from the width and depth of deep architectures

From shallow feature learning to deep learning: Benefits from the width and depth of deep architectures

Towards Improved Classification Accuracy on Highly Imbalanced Text Dataset Using Deep Neural Language Models

Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics

Contact Info

Product

Resources

About