Enabling Hierarchical Dirichlet Processes to Work Better for Short Texts at Large Scale

Mai, Khai; Mai, Sang; Nguyen, Anh Tung; Linh, Ngo Van; Than, Khoat

doi:10.1007/978-3-319-31750-2_34

Cited by 5 publications

(3 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Long texts: We use two large datasets of long texts for evaluation: PubMed dataset consists of 330,000 articles from the PubMed central and New York Times dataset consists of 300,000 news 2 . • Short texts: We use three large datasets of short texts for evaluation: Yahoo Questions crawled from answers.yahoo.com, each document is a question; Tweets from Twitter (twitter.com), each document is the text content of a tweet; NYT-Titles from The New York Times (www.nytimes.com), each document is the title of an article [58]. These datasets are preprocessed by tokenizing, stemming, removing stop-words, removing low-frequency words (appear in less than 3 documents) and removing extremely short documents (less than 3 words).…”

Section: B Empirical Evaluationmentioning

confidence: 99%

“…These datasets are preprocessed by tokenizing, stemming, removing stop-words, removing low-frequency words (appear in less than 3 documents) and removing extremely short documents (less than 3 words). The shortness of texts poses various difficulties [58]- [60] because of its natural characters such as sparseness, largescale, immediacy, non-standardization [57]. It is difficult for traditional methods to deal with short texts mainly because too limited words in the short text cannot represent the feature space and the relationship between words and documents.…”

Section: B Empirical Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

MAP Estimation With Bernoulli Randomness, and Its Application to Text Analysis and Recommender Systems

Bui

Nguyen

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

MAP estimation plays an important role in many probabilistic models. However, in many cases, the MAP problem is non-convex and intractable. In this work, we propose a novel algorithm, called BOPE, which uses Bernoulli randomness for Online Maximum a Posteriori Estimation. We show that BOPE has a fast convergence rate. In particular, BOPE implicitly employs a prior which plays as regularization. Such a prior is different from the one of the MAP problem and will be vanishing as BOPE does more iterations. This property of BOPE is significant and enables to reduce severe overfitting for probabilistic models in ill-posed cases, including short text, sparse data, and noisy data. We validate the practical efficiency of BOPE in two contexts: text analysis and recommender systems. Both contexts show the superior of BOPE over the baselines.

show abstract

Section: B Empirical Evaluationmentioning

confidence: 99%

Section: B Empirical Evaluationmentioning

confidence: 99%

MAP Estimation With Bernoulli Randomness, and Its Application to Text Analysis and Recommender Systems

Bui

Nguyen

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

“…Simultaneously, it should be plastic when concept drift happens. Secondly, noisy and sparse data (Nguyen et al, 2021;Ha et al, 2019;Mai et al, 2016;Tuan et al, 2020) makes a lot of difficulties for learning methods. While sparse data does not provide an unclear context, noisy data can mislead the methods.…”

Section: Introductionmentioning

confidence: 99%

Adaptive infinite dropout for noisy and sparse data streams

et al. 2022

Self Cite

View full text Add to dashboard Cite

The ability to analyze data streams, which arrive sequentially and possibly infinitely, is increasingly vital in various online applications. However, data streams pose various challenges, including sparse and noisy data as well as concept drifts, which easily mislead a learning method. This paper proposes a simple yet robust framework, called Adaptive Infinite Dropout (aiDropout), to effectively tackle these problems. Our framework uses a dropout technique in a recursive Bayesian approach in order to create a flexible mechanism for balancing between old and new information. In detail, the recursive Bayesian approach imposes a constraint on the model parameters to make a regularization term between the current and previous mini-batches. Then, dropout whose drop rate is autonomously learned can adjust the constraint to new data. Thanks to the ability to reduce overfitting and the ensemble property of Dropout, our framework obtains better generalization, thus it effectively handles undesirable effects of noise and sparsity. In particular, theoretical analyses show that aiDropout imposes a data-dependent regularization, therefore, it can adapt quickly to sudden changes from data streams. Extensive experiments show that aiDropout significantly outperforms the state-of-the-art baselines on a variety of tasks such as supervised and unsupervised learning.

show abstract