Document classification using term frequency-inverse document frequency and K-means clustering

Abstract

Increased advancement in a variety of study subjects and information technologies, has increased the number of published research articles. However, researchers are facing difficulties and devote a significant time amount in locating scientific research publications relevant to their domain of expertise. In this article, an approach of document classification is presented to cluster the text documents of research articles into expressive groups that encompass a similar scientific field. The main focus and scopes of target groups were adopted in designing the proposed method, each group include several topics. The word tokens were separately extracted from topics related to a single group. The repeated appearance of word tokens in a document has an impact on the document's weight, which is computed using the term frequency-inverse document frequency (TF-IDF) numerical statistic. To perform the categorization process, the proposed approach employs the paper's title, abstract, and keywords, as well as the categories' topics. We exploited the K-means clustering algorithm for classifying and clustering the documents into primary categories. The K-means algorithm uses category weights to initialize the cluster centers (or centroids). Experimental results have shown that the suggested technique outperforms the k-nearest neighbors algorithm in terms of accuracy in retrieving information.

Exploration of the Thematic Clustering and Collaboration Opportunities in Kazakhstani Research

“…The Porter Stemming algorithm, for instance, employs a series of about 60 rules applied sequentially. Each rule is of the form: (1) where S1 is a suffix to be replaced by S2 if a condition (usually related to the measure of the stem, or m is satisfied. The measure m is calculated as:…”

“…Al-Obaydy, Hashim, Najm and Jalal propose an innovative approach for categorizing research articles into thematic groups, leveraging Term Frequency-Inverse Document Frequency (TF-IDF) and K-means clustering. The methodology is designed to address the challenges researchers face in navigating the vast corpus of scientific literature, aiming to cluster text documents into meaningful groups that represent similar scientific fields [1]. Shetty and Kallimani introduce an innovative approach leveraging K-Means clustering for extractive text summarization, focusing on preserving semantic richness while eliminating redundancy [2].…”

Algoritma K-Means Clustering Penggunaan Bandwidth Internet (Studi Kasus di Pemerintah Daerah Kabupaten Padang Pariaman)

“…Algoritma K-means memberikan metode sederhana untuk mengeksekusi solusi perkiraan [11]. Ini adalah pengelompokan eksklusif dan salah satu algoritma yang paling banyak digunakan untuk pengelompokan [12]. Algoritma ini sudah banyak digunakan pada penelitian sebelumnya [13].…”

Efficient information retrieval model: overcoming challenges in search engines-an overview

“…The Okapi BM25 model [10] is a popular probabilistic model that uses criteria such as term frequency, document length, and document frequency to compute the relevance score of a document [11], [12]. Reinforcement learning Reinforcement learning is a type of machine learning that involves training an agent to learn by trial and error.…”