Generative Adversarial Networks for Text Using Word2vec Intermediaries

Budhkar, Akshay; Vishnubhotla, Krishnapriya; Hossain, Safwan; Rudzicz, Frank

doi:10.18653/v1/w19-4303

Cited by 4 publications

(4 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, there are many specific tricks that can accelerate the training of GAN. In addition to the way mentioned in Section 4.6.2, one can see Bose, Ling, and Cao (2018) and Budhkar et al (2019) for more suggestions.…”

Section: Further Discussion C1 Limitation: Time Complexitymentioning

confidence: 99%

“…GAN-based negative sampler has also been used in other lines of NLP research (Wang, Liu, and Zhao, 2017;Wang, Li, and Pan, 2018;Dai et al, 2019;Budhkar et al, 2019). For example, Wang, Li, and Pan (2018) employed GAN-based framework on knowledge representation learning and Budhkar et al (2019) imposed GAN-based method to generate natural language.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

On SkipGram Word Embedding Models with Negative Sampling: Unified Framework and Impact of Noise Distributions

Wang,

Mao,

Guo

et al. 2020

Preprint

View full text Add to dashboard Cite

SkipGram word embedding models with negative sampling, or SGN in short, is an elegant family of word embedding models. In this paper, we formulate a framework for word embedding, referred to as Word-Context Classification (WCC), that generalizes SGN to a wide family of models. The framework, utilizing some "noise examples", is justified through a theoretical analysis. The impact of noise distribution on the learning of the WCC embedding models is studied experimentally, suggesting that the best noise distribution is in fact the data distribution, in terms of both the embedding performance and the speed of convergence during training. Along our way, we discover several novel embedding models that outperform the existing WCC models.

show abstract

Section: Further Discussion C1 Limitation: Time Complexitymentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

On SkipGram Word Embedding Models with Negative Sampling: Unified Framework and Impact of Noise Distributions

Wang,

Mao,

Guo

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The discontinuous nature of the text is handled using a GAN2vec technique. The experimental GAN2vec evaluation on the dataset of Chinese poetry yields a BLUE score of 66.08% (Budhkar et al 2019). An ensemble approach is suggested to classify brief text sequences from the texts of various Arabic-speaking nations.…”

Section: Named Entity Recognition and Recommendation Systemmentioning

confidence: 99%

Impact of word embedding models on text analytics in deep learning environment: a review

2023

View full text Add to dashboard Cite

The selection of word embedding and deep learning models for better outcomes is vital. Word embeddings are an n-dimensional distributed representation of a text that attempts to capture the meanings of the words. Deep learning models utilize multiple computing layers to learn hierarchical representations of data. The word embedding technique represented by deep learning has received much attention. It is used in various natural language processing (NLP) applications, such as text classification, sentiment analysis, named entity recognition, topic modeling, etc. This paper reviews the representative methods of the most prominent word embedding and deep learning models. It presents an overview of recent research trends in NLP and a detailed understanding of how to use these models to achieve efficient results on text analytics tasks. The review summarizes, contrasts, and compares numerous word embedding and deep learning models and includes a list of prominent datasets, tools, APIs, and popular publications. A reference for selecting a suitable word embedding and deep learning approach is presented based on a comparative analysis of different techniques to perform text analytics tasks. This paper can serve as a quick reference for learning the basics, benefits, and challenges of various word representation approaches and deep learning models, with their application to text analytics and a future outlook on research. It can be concluded from the findings of this study that domain-specific word embedding and the long short term memory model can be employed to improve overall text analytics task performance.

show abstract

“…Common pretrained word embedding techniques include Word2Vec [102,103], GloVE [104], and fastText [105]. Word embedding representations are often used as input to models that tackle NLP tasks, such as sentiment analysis (e.g., a CNN with Word2vec and GloVE in [106], a CNN with Word2Vec in [107]), text classification (e.g., Support Vector Machines (SVM) [108] accompanied with Word2Vec, TF-IDF in [109]), question answering (e.g., a dependency-tree recurrent neural network with Word2Vec in [110]), text generation (e.g., Generative Adversarial Networks with Word2Vec in [111]).…”

Section: Text Modalitymentioning

confidence: 99%

AttendAffectNet–Emotion Prediction of Movie Viewers Using Multimodal Fusion with Self-Attention

Thao

Balamurali

Roig

et al. 2021

Sensors

View full text Add to dashboard Cite

In this paper, we tackle the problem of predicting the affective responses of movie viewers, based on the content of the movies. Current studies on this topic focus on video representation learning and fusion techniques to combine the extracted features for predicting affect. Yet, these typically, while ignoring the correlation between multiple modality inputs, ignore the correlation between temporal inputs (i.e., sequential features). To explore these correlations, a neural network architecture—namely AttendAffectNet (AAN)—uses the self-attention mechanism for predicting the emotions of movie viewers from different input modalities. Particularly, visual, audio, and text features are considered for predicting emotions (and expressed in terms of valence and arousal). We analyze three variants of our proposed AAN: Feature AAN, Temporal AAN, and Mixed AAN. The Feature AAN applies the self-attention mechanism in an innovative way on the features extracted from the different modalities (including video, audio, and movie subtitles) of a whole movie to, thereby, capture the relationships between them. The Temporal AAN takes the time domain of the movies and the sequential dependency of affective responses into account. In the Temporal AAN, self-attention is applied on the concatenated (multimodal) feature vectors representing different subsequent movie segments. In the Mixed AAN, we combine the strong points of the Feature AAN and the Temporal AAN, by applying self-attention first on vectors of features obtained from different modalities in each movie segment and then on the feature representations of all subsequent (temporal) movie segments. We extensively trained and validated our proposed AAN on both the MediaEval 2016 dataset for the Emotional Impact of Movies Task and the extended COGNIMUSE dataset. Our experiments demonstrate that audio features play a more influential role than those extracted from video and movie subtitles when predicting the emotions of movie viewers on these datasets. The models that use all visual, audio, and text features simultaneously as their inputs performed better than those using features extracted from each modality separately. In addition, the Feature AAN outperformed other AAN variants on the above-mentioned datasets, highlighting the importance of taking different features as context to one another when fusing them. The Feature AAN also performed better than the baseline models when predicting the valence dimension.

show abstract

Generative Adversarial Networks for Text Using Word2vec Intermediaries

Cited by 4 publications

References 20 publications

On SkipGram Word Embedding Models with Negative Sampling: Unified Framework and Impact of Noise Distributions

On SkipGram Word Embedding Models with Negative Sampling: Unified Framework and Impact of Noise Distributions

Impact of word embedding models on text analytics in deep learning environment: a review

AttendAffectNet–Emotion Prediction of Movie Viewers Using Multimodal Fusion with Self-Attention

Contact Info

Product

Resources

About