A Practical and Empirical Comparison of Three Topic Modeling Methods Using a COVID-19 Corpus: LSA, LDA, and Top2Vec

Abstract

This study was prepared as a practical guide for researchers interested in using topic modeling methodologies. This study is specially designed for those with difficulty determining which methodology to use. Many topic modeling methods have been developed since the 1980s namely, latent semantic indexing or analysis (LSI/LSA), , probabilistic LSI/LSA (pLSI/pLSA), naïve Bayes, the Author-Recipient-Topic (ART), Latent Dirichlet Allocation (LDA), Topic Over Time (TOT), Dynamic Topic Models (DTM), Word2Vec, Top2Vec and \variation and combination of these techniques. For researchers from disciplines other than computer science may find it challenging to select a topic modeling methodology. We compared a recently developed topic modeling algorithm Top2Vec with two of the most conventional and frequently-used methodologiesLSA and LDA. As a study sample, we used a corpus of 65,292 COVID-19-focused abstracts. Among the 11 topics we identified in each methodology, we found high levels of correlation between LDA and Top2Vec results, followed by LSA and LDA and Top2Vec and LSA. We also provided information on computational resources we used to perform the analyses and provided practical guidelines and recommendations for researchers.

Topic Modeling for Scientific Articles: Exploring Optimal Hyperparameter Tuning in BERT

“…The algebraic topic model, such as Latent Semantic Allocation (LSA), was developed in the 1990s [2], represents the corpus as a document term matrix (DTM). Zengul et al [12]state that LSA and topic modeling are among the most commonly employed methods. LSA is a natural language processing approach that examines associations between text-based terms and documents, assuming that words with similar meanings will occur in similar contexts.…”