Can end‐user feedback in social media be trusted for software evolution: Exploring and analyzing fake reviews

Abstract

SummaryEnd‐user feedback in social media platforms, particularly in the app stores, is increasing exponentially with each passing day. Software researchers and vendors started to mine end‐user feedback by proposing text analytics methods and tools to extract useful information for software evolution and maintenance. In addition, research shows that positive feedback and high‐star app ratings attract more users and increase downloads. However, it emerged in the fake review market, where software vendors started incorporating fake reviews against their corresponding applications to improve overall software ratings. For this purpose, we conducted an exploratory study to understand how end‐users register and write fake reviews in the Google Play Store. We curated a research data set containing 68,000 end‐user comments from the Google Play Store and a fake review generator, that is, the Testimonial generator (TG). Its purpose is to understand fake reviews on these platforms and identify the common patterns potential end‐users and professionals use to report fake reviews by critically analyzing the end‐user feedback. We conducted a detailed survey at the University of Science and Technology Bannu, Pakistan, to identify the intelligence and accuracy of crowd‐users in manually identifying fake reviews. In addition, we developed a ground truth to be compared with the results obtained from the automated machine and deep learning (M&DL) classifier experiment. In the survey, 512 end‐users participated and recorded their responses in identifying fake reviews. Finally, various M&DL classifiers are employed to classify and identify end‐user reviews into real and fake to automate the process. Unlike humans, the M&DL classifiers performed well in automatically classifying reviews into real and fake by obtaining much higher accuracy, precision, recall, and f‐measures. The accuracy of manually identifying fake reviews by the crowd‐users is 44.4%. In contrast, the M&DL classifiers obtained an average accuracy of 96%. The experimental results obtained with various M&DL classifiers are encouraging. It is the first step towards identifying fake reviews in the app store by studying its implications in software and requirements engineering.