Empirical evaluation and study of text stemming algorithms

Jabbar, Abdul; Iqbal, Sajid; Tamimy, Manzoor Ilahi; Hussain, Shafiq; Akhunzada, Adnan

doi:10.1007/s10462-020-09828-3

Cited by 31 publications

(14 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…NLP is regarded as a sub-domain of 'dialectology', 'computer science', 'knowledge engineering' and 'AI' associating fundamental relationships between computers and humane dialects (Jabbar et al, 2020). For the most part, it determined systems to process and analyse enormous natural language data.…”

Section: Natural Language Processingmentioning

confidence: 99%

Realization of natural language processing and machine learning approaches for text‐based sentiment analysis

Naithani

Raiwani

2022

Expert Systems

View full text Add to dashboard Cite

The leading intention of the current paper is to review the research work accomplished by various researchers to achieve sentiment analysis on the text and to elaborate on natural language processing (NLP) and various machine learning algorithms used to evaluate textual sentiments. In this study, primitive cases are considered that used crucial algorithms, and knowledge that can be opted for sentiment analysis. A survey of the work that has been done till now is conducted observing the results and outcomes concerning varying parameters of various researchers who worked on previously existing as well as novel and hybrid algorithms opting legion methodologies. The fundamental algorithms like Support Vector Machine (SVM), Bayesian Networks (BN), Maximum Entropy (MaxEnt), Conditional Random Fields (CRF) and Artificial Neural Networks (ANN) are also discussed to achieve practice percentage and accuracy score in the field of NLP, sentiment analysis and text analytics. Various other novel approaches and algorithms like CNN, LSTM, KNN, K*, K-means, K-means++, SOM and ENORA, along with their limitations and the performance metrics providing accuracies for major open data sets are also analyzed.

show abstract

Section: Natural Language Processingmentioning

confidence: 99%

Realization of natural language processing and machine learning approaches for text‐based sentiment analysis

Naithani

Raiwani

2022

Expert Systems

View full text Add to dashboard Cite

show abstract

“…There are various stemming algorithms forever in every language, such as Potter's algorithm for English word stemming. Jabbar et al [24] discusses various stemming algorithms used to stem textual data.…”

Section: ) Stemmingmentioning

confidence: 99%

Subjective Answers Evaluation Using Machine Learning and Natural Language Processing

Bashir

Arshad

Javed³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Subjective paper evaluation is a tricky and tiresome task to do by manual labor. Insufficient understanding and acceptance of data are crucial challenges while analyzing subjective papers using Artificial Intelligence (AI). Several attempts have been made to score students' answers using computer science. However, most of the work uses traditional counts or specific words to achieve this task. Furthermore, there is a lack of curated data sets as well. This paper proposes a novel approach that utilizes various machine learning, natural language processing techniques, and tools such as Wordnet, Word2vec, word mover's distance (WMD), cosine similarity, multinomial naive bayes (MNB), and term frequencyinverse document frequency (TF-IDF) to evaluate descriptive answers automatically. Solution statements and keywords are used to evaluate answers, and a machine learning model is trained to predict the grades of answers. Results show that WMD performs better than cosine similarity overall. With enough training, the machine learning model could be used as a standalone as well. Experimentation produces an accuracy of 88% without the MNB model. The error rate is further reduced by 1.3% using MNB.

show abstract

“…Many stemming algorithms have been developed for different languages up to date. [11][12][13][14][15] The first of these algorithms is the one published by Lovins in 1968. 8 The most used one today is the Porter stemming algorithm.…”

Section: Related Workmentioning

confidence: 99%

“…Stemming could be used to extract information from sources like big data or the Internet since additional forms of a word related to desired subject would also be searched for best results. Many stemming algorithms have been developed for different languages up to date 11‐15 . The first of these algorithms is the one published by Lovins in 1968 8 .…”

Section: Related Workmentioning

confidence: 99%

A framework for investigating search engines' stemming mechanisms: A case study on Bing

Şentürk

Gunduz

2021

Concurrency and Computation

View full text Add to dashboard Cite

Big data attracts the attention of governments and a lot of companies today. The developments in technology and the Internet make it one of the important sources of big data. It is easy to get lost in the enormous amount of information contained on the Internet if there were no search engines. Knowing how the search engines work will be helpful to access the desired information. This work aims to be a guide for accessing the right information and also to help to understand search engine stemming and indexing algorithm for interested parties. In this article, we have developed a framework that could be used to investigate the stemming mechanisms of search engines. Our framework also uses Word2vec to analyze semantic relations. We have used our framework to investigate the stemming algorithm of the search engine Bing for English language.In order to achieve that we have used this framework to select words, create queries, send them to Bing, and finally analyze the millions of returned results. We have discussed the results in the context of our article. The results indicate that our framework is useful for analyzing the stemming mechanisms of search engines.

show abstract

Empirical evaluation and study of text stemming algorithms

Cited by 31 publications

References 51 publications

Realization of natural language processing and machine learning approaches for text‐based sentiment analysis

Realization of natural language processing and machine learning approaches for text‐based sentiment analysis

Subjective Answers Evaluation Using Machine Learning and Natural Language Processing

A framework for investigating search engines' stemming mechanisms: A case study on Bing

Contact Info

Product

Resources

About