Statistics-Based Outlier Detection and Correction Method for Amazon Customer Reviews

Abstract

People nowadays use the internet to project their assessments, impressions, ideas, and observations about various subjects or products on numerous social networking sites. These sites serve as a great source to gather data for data analytics, sentiment analysis, natural language processing, etc. Conventionally, the true sentiment of a customer review matches its corresponding star rating. There are exceptions when the star rating of a review is opposite to its true nature. These are labeled as the outliers in a dataset in this work. The state-of-the-art methods for anomaly detection involve manual searching, predefined rules, or traditional machine learning techniques to detect such instances. This paper conducts a sentiment analysis and outlier detection case study for Amazon customer reviews, and it proposes a statistics-based outlier detection and correction method (SODCM), which helps identify such reviews and rectify their star ratings to enhance the performance of a sentiment analysis algorithm without any data loss. This paper focuses on performing SODCM in datasets containing customer reviews of various products, which are (a) scraped from Amazon.com and (b) publicly available. The paper also studies the dataset and concludes the effect of SODCM on the performance of a sentiment analysis algorithm. The results exhibit that SODCM achieves higher accuracy and recall percentage than other state-of-the-art anomaly detection algorithms.

Natural Language Processing for Analyzing Online Customer Reviews: A Survey, Taxonomy, and Open Research Challenges

“…A unique Statistics-Based Outlier Detection and Correction Method study [34] highlighted the need for proper sentiment analysis in Amazon customer reviews. This technology improved sentiment analysis without data loss over previous systems.…”

“…[33] -Investigated sentiment analysis on WWW content, utilizing a lexicon-based method and logistic regression in machine learning. [34] Amazon Investigated sentiment analysis and outlier detection in Amazon customer reviews. [35] Amazon Focused on sentiment analysis of Amazon electronics product reviews.…”

A New Particle Swarm Optimization Algorithm for Outlier Detection: Industrial Data Clustering in Wire Arc Additive Manufacturing

“…Serving as a popular data analysis method, outlier detection plays an important role in identifying abnormal instances [10]. In the past few decades, a lot of outlier detection methods have been proposed [1], [13], [40], [45], [56]. For example, machine-learning-based outlier detection methods have been employed in [40] to analyze the semiconductor manufacturing etching data.…”

“…1,k and c 2,k are the acceleration coefficients; r 1 and r 2 represent random numbers selected within [0, 1]; pbest i,k represents the personal best position found by the ith particle itself at the kth iteration; gbest k represents the global best position of the entire swarm at the kth iteration; α 1,ξ k and α 2,ξ k are parameters which are used to adjust the Gaussian white noises according to the evolutionary state; and δ 1 and δ 2 represent two independent Gaussian white noises.The procedure of the proposed ASRPPSO is presented in Algorithm 1. Calculate each particle's fitness value.…”

Emotional Variance Analysis: A new sentiment analysis feature set for Artificial Intelligence and Machine Learning applications

“…In this study, we address the above drawbacks by expanding calculations of sentiment polarity [ 11 , 24 , 25 ], orientation of the expressed sentiment (positive, negative or neutral) and sentiment polarity shifts–which can help capture significant emotional changes [ 26 ]. Our proposed feature set, Emotional Variance Analysis (EVA), captures and profiles changes in sentiment polarity and intensity to accurately classify emotional instability [ 10 , 15 , 20 ]. EVA comprises 21 novel EVA features calculated from extracted absolute rankings and sentiment scores based on observations of sentiment polarity shift profiles as relative polarity changes, through word-sentence vocabulary structures.…”