A Suggestion on the LDA-Based Topic Modeling Technique Based on ElasticSearch for Indexing Academic Research Results

Mi, Kim Young; Kim, Do‐Sung

doi:10.3390/app12063118

Cited by 8 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the realm of computer science, Natural Language Processing (NLP) represents a challenging and vital field of study that empowers computers to comprehend human language within textual documents. Within this context, topic modeling approaches stand out as robust and intelligent algorithms frequently harnessed in NLP to uncover topics and extract valuable insights from unstructured text documents [22]. Broadly speaking, topic modeling techniques, particularly those grounded in Latent Dirichlet Allocation (LDA) discover many uses for text mining, social media analysis, and information retrieval and the broader domain of natural language processing.…”

Section: Topic Modelingmentioning

confidence: 99%

Enhancing Data Lake Management Systems with LDA Approach

Cherradi,

Haddadi

2024

JDSIS

View full text Add to dashboard Cite

In today’s fiercely competitive business landscape, data has emerged as a precious asset crucial for any company’s growth. It embodies a genuine catalyst for economic and strategic advantages, distinguishing industry leaders from the rest. Prominent organizations recognize the importance of not just amassing data from diverse sources but also harnessing the transformative power of data analytics for informed determinations processes. Within this setting, the data lake solution stands as a robust framework handling vast data sources and enabling data investigations to support decision-making tasks. This paper delves into the realm of intelligent data lake management systems designed to overcome the limitations of traditional business intelligence, which struggles to meet the demands of data-driven decision-making. Data lakes excel in the analysis of data from myriad sources, particularly when data cleaning becomes a time-consuming endeavor. Still, managing diverse datasets devoid of a predefined data structure presents a significant challenge, potentially leading to a data lake devolving to a data swamp. Within this article, we adopt the Latent Dirichlet Allocation model to oversee the data lake environment's handling, processing, analysis, and display of huge datasets. To evaluate the efficacy of our suggested approach, we conducted comprehensive assessments using the topic coherence metric. Our experiments yielded results indicating the superior accuracy of our approach when applied to the tested datasets.

show abstract

Section: Topic Modelingmentioning

confidence: 99%

Enhancing Data Lake Management Systems with LDA Approach

Cherradi,

Haddadi

2024

JDSIS

View full text Add to dashboard Cite

show abstract

“…In their research, Kim et al [22] combined a classification system based on ElasticSearch and topic modelling based on Latent Dirichlet Allocation to simplify searching for academic research results in online literature databases. As a result, several research topics, such as predictive analysis, learning processes, and data classification, are trending topics that often appear in current academic research.…”

Section: Related Workmentioning

confidence: 99%

Leveraging Topic Modelling to Analyze Biomedical Research Trends from the PubMed Database Using LDA Method

Pamungkas

2024

SISFOKOM

View full text Add to dashboard Cite

Biomedical research has become an essential entity in human life. However, finding trends related to research topics in the health sector contained in the repository is a challenging matter. In this study, we implemented topic modelling to analyze biomedical research trends using the LDA method. Topic modelling was carried out using data from 7000 articles from PubMed, which were processed with text processing such as lowercase, punctuation removal, tokenization, stop-word removal, and lemmatization. For topic modelling, the LDA with corpus conditions varied to 75% and 100% for validation. Alpha and beta parameters are also set with variations between 0.01, 0.31, 0.61, 0.91, symmetry, and asymmetry when the number of the corpus is changed. When the number of the corpus is 75%, the optimal number of topics is 7, with a coherence value of 0.52. Whereas when the number of the corpus is 100%, the optimal number of topics is 10 with a coherence value of 0.51. In addition, based on the results of article topic modelling, several topics are trending, including disease diagnosis, patient care, and genetic or cell research. Based on the classification of biomedical topics into seven categories, the optimal accuracy, precision, and recall values using the Random Forest algorithm were obtained, namely 85.57%, 87.36%, and 87.58%. The results of this study suggest that topic modelling using the LDA can be used to identify trends in biomedical research with high accuracy. This information can help stakeholders make informed decisions about the direction of future research.

show abstract

“…(9,11) Since 2016, the popularity of data lakes has exploded in both the business and academic communities. There are high-level proposals about data lake architecture, (2,3,4,11) as well as comparisons with data warehouses, (12) and publications that discuss its concept, components, and issues. However, a number of companies have created data lakes as commercial solutions, including IBM and Cloudera, Google, Microsoft, Azure, SAP, Amazon AWS, Snowflake, and Oracle.…”

Section: Data Lakementioning

confidence: 99%

“…(1) By finding new possibilities, highlighting possible pitfalls, uncovering new business insights, and improving decision-making processes, business intelligence helps organizations to increase their productivity. (2) As a result, most industries consider BI to be a high priority. However, conventional business intelligence concentrates on structured data only, ignoring very important information hidden in unstructured data.…”

Section: Introductionmentioning

confidence: 99%

Data Lake Management System based on Topic Modeling

El Haddadi,

Cherradi

et al. 2023

Data and Metadata

View full text Add to dashboard Cite

In an environment full of competitiveness, data is a valuable asset for any company looking to grow. It represents a real competitive economic and strategic lever. The most reputable companies are not only concerned with collecting data from heterogeneous data sources, but also with analyzing and transforming these datasets into better decision-making. In this context, the data lake continues to be a powerful solution for storing large amounts of data and providing data analytics for decision support. In this paper, we examine the intelligent data lake management system that addresses the drawbacks of traditional business intelligence, which is no longer capable of handling data-driven demands. Data lakes are highly suitable for analyzing data from a variety of sources, particularly when data cleaning is time-consuming. However, ingesting heterogeneous data sources without any schema represents a major issue, and a data lake can easily turn into a data swamp. In this study, we implement the LDA topic model for managing the storage, processing, analysis, and visualization of big data. To assess the usefulness of our proposal, we evaluated its performance based on the topic coherence metric. The results of these experiments showed our approach to be more accurate on the tested datasets.

show abstract

A Suggestion on the LDA-Based Topic Modeling Technique Based on ElasticSearch for Indexing Academic Research Results

Cited by 8 publications

References 23 publications

Enhancing Data Lake Management Systems with LDA Approach

Enhancing Data Lake Management Systems with LDA Approach

Leveraging Topic Modelling to Analyze Biomedical Research Trends from the PubMed Database Using LDA Method

Data Lake Management System based on Topic Modeling

Contact Info

Product

Resources

About