Binary Autoencoder for Text Modeling

Baynazarov, Ruslan; Piontkovskaya, Irina

doi:10.1007/978-3-030-34518-1_10

Cited by 8 publications

(9 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finding noisy data in various datasets has always been of importance in traditional machine learning [3] and deep learning [17] . Using Autoencoders is the most popular approach among all and it has also been useful in natural language processing [8] . To obtain the distribution of the data before the actual event, the output documents of the Sentence-BERT model are used to train the autoencoder network.…”

Section: Distributional Denoising Autoencodermentioning

confidence: 99%

A Semantic Modular Framework for Events Topic Modeling in Social Media

Moghaddam¹,

Momtazi²

2023

Preprint

View full text Add to dashboard Cite

The advancement of social media contributes to the growing amount of content they share frequently. This framework provides a sophisticated place for people to report various real-life events. Detecting these events with the help of natural language processing has received researchers' attention, and various algorithms have been developed for this goal. In this paper, we propose a Semantic Modular Model (SMM) consisting of 5 different modules, namely Distributional Denoising Autoencoder, Incremental Clustering, Semantic Denoising, Defragmentation, and Ranking and Processing. The proposed model aims to (1) cluster various documents and ignore the documents that might not contribute to the identification of events, (2) identify more important and descriptive keywords. Compared to the state-of-the-art methods, the results show that the proposed model has a higher performance in identifying events with lower ranks and extracting keywords for more important events in three English Twitter datasets: FACup, SuperTuesday, and USElection. The proposed method outperformed the best reported results in the mean keyword-precision metric by 7.9%.

show abstract

Section: Distributional Denoising Autoencodermentioning

confidence: 99%

A Semantic Modular Framework for Events Topic Modeling in Social Media

Moghaddam¹,

Momtazi²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Variational Autoencoders [ 27 ] (VAEs) play an important role in text generation tasks, when semantically consistent latent space is needed; however, VAEs training generally suffers from mode collapse issues. The authors of [ 28 ] developed an autoencoder with binary latent space using a straight-through estimator: experiments showed that this approach maintains the main features of VAE, e.g., semantic consistency and good latent space coverage, while not suffering from the mode collapse, other than being much easier to train. One of the most successful uses of autoencoders is for noise removal.…”

Section: Related Workmentioning

confidence: 99%

“…Autoencoders are another successful deep architectures where the aim is to reconstruct a signal by learning a latent representation from a set of data. They have been used for realistic text [20] and images [21] generation; however, one of the most successful use of autoencoders is for noise removal. Since their introduction [22], denoising autoencoders (DAE) have been used for a broad number of tasks like medical images improvement [23], speech enhancement [24] and ECG signal boost [25].…”

Section: Related Workmentioning

confidence: 99%

DANAE++: A Smart Approach for Denoising Underwater Attitude Estimation

Russo

Ciaccio

Troisi

2021

Sensors

View full text Add to dashboard Cite

One of the main issues for the navigation of underwater robots consists in accurate vehicle positioning, which heavily depends on the orientation estimation phase. The systems employed to this end are affected by different noise typologies, mainly related to the sensors and to the irregular noise of the underwater environment. Filtering algorithms can reduce their effect if opportunely configured, but this process usually requires fine techniques and time. This paper presents DANAE++, an improved denoising autoencoder based on DANAE (deep Denoising AutoeNcoder for Attitude Estimation), which is able to recover Kalman Filter (KF) IMU/AHRS orientation estimations from any kind of noise, independently of its nature. This deep learning-based architecture already proved to be robust and reliable, but in its enhanced implementation significant improvements are obtained in terms of both results and performance. In fact, DANAE++is able to denoise the three angles describing the attitude at the same time, and that is verified also using the estimations provided by an extended KF. Further tests could make this method suitable for real-time applications in navigation tasks.

show abstract

“…proposed by [21], [22], and the authors of [23] relax the discrete latents and durations of a recurrent hidden semi-Markov Model. Applications of (Gumbel-based) discrete latent variable models as described above include (among others) planning [24], syntactic parsing [25], text modelling [26], speech modelling [27], [28], paraphrase generation [29], recommender systems [30], drug-drug interaction modelling [31], and event modelling [32].…”

Section: Discrete Latent Variable Modelsmentioning

confidence: 99%

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

Huijben¹,

Kool²,

Paulus³

et al. 2021

Preprint

View full text Add to dashboard Cite

The Gumbel-max trick is a method to draw a sample from a categorical distribution, given by its unnormalized (log-)probabilities. Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation for error backpropagation in neural network optimization. The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. Moreover, it presents a comprehensive outline of (machine learning) literature in which Gumbel-based algorithms have been leveraged, reviews commonly-made design choices, and sketches a future perspective.

show abstract

Binary Autoencoder for Text Modeling

Cited by 8 publications

References 6 publications

A Semantic Modular Framework for Events Topic Modeling in Social Media

A Semantic Modular Framework for Events Topic Modeling in Social Media

DANAE++: A Smart Approach for Denoising Underwater Attitude Estimation

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

Contact Info

Product

Resources

About