Preprocessing Techniques in Text Categorization: A Survey

Malik, Sayyam; Sani, Sana Ahmad; Baqir, Anees; Ahmad, Usman; Mustafa, Faizan ul

doi:10.1007/978-981-15-5232-8_43

Cited by 2 publications

(1 citation statement)

References 2 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• Tokenization: Tokenization (Malik et al, 2019) is the process which breakdown the larger stringintosmallerone.Themaingoaloftokenizationistotokenizebiggerpiecesofcontent intosentences,andsentencesaretokenizingintowords,andsoon.…”

Section: Related Workmentioning

confidence: 99%

Prospecting the Effect of Topic Modeling in Information Retrieval

Sharaff

Dewangan

Sisodia

2021

International Journal on Semantic Web and Information Systems

View full text Add to dashboard Cite

Enormous records and data are gathered every day. Organization of this data is a challenging task. Topic modeling provides a way to categorize these documents, where high dimensionality of the corpus affects the result of topic model, making it important to apply feature selection or information retrieval process for dimensionality reduction. The requirement for efficient topic modeling includes the removal of unrelated words that might lead to specious coexistence of the unrelated words. This paper proposes an efficient framework for the generation of better topic coherence, where term frequency-inverse document frequency (TF-IDF) and parsimonious language model (PLM) are used for the information retrieval task. PLM extracts the important information and expels the general words from the corpus, whereas TF-IDF re-estimates the weightage of each word in the corpus. The work carried out in this paper improved the topic coherence measure to provide a better correlation among the actual topic and the topics generated from PLM.

show abstract

Section: Related Workmentioning

confidence: 99%