What's in a Review

Abstract

As video game press ("experts") and casual gamers ("amateurs") have different motivations when writing video game reviews, discrepancies in their reviews may arise. To study such potential discrepancies, we conduct a large-scale investigation of more than 1 million reviews on the Metacritic review platform. In particular, we assess the existence and nature of discrepancies in video game appraisal by experts and amateurs, and how they manifest in ratings, over time, and in review language. Leveraging these insights, we explore the predictive power of early expert vs. amateur reviews in forecasting video game reputation in the short-and long-term. We find that amateurs, in contrast to experts, give more polarized ratings of video games, rate games surprisingly long after game release, and are positively biased towards older games. On a textual level, we observe that experts write rather complex, less readable texts than amateurs, whose reviews are more emotionally charged. While in the short-term amateur reviews are remarkably predictive of game reputation among other amateurs (achieving 91% ROC AUC in a binary classification), both expert and amateur reviews are equally well suited for long-term predictions. Overall, our work is the first large-scale comparative study of video game reviewing behavior, with practical implications for amateurs when deciding which games to play, and for game developers when planning which games to design, develop, or continuously support. More broadly, our work contributes to the discussion of wisdom of the few vs. wisdom of the crowds, as we uncover the limits of experts in capturing the views of amateurs in the particular context of video game reviews.

The Impacts of Online Reviews on Consumers Purchase Intention of Video Game Items in China

“…Moreover, by testing the effects of different factors, including valance, volume, and consistency of reviews, researchers also identified influencer reviews as the determinant of game sales [5][6]. Other research also demonstrates that influencer reviews could predict the video game's reputation in the long term [7].…”

“…user-generated content is expected to be more authentic and self-expressive than mass-media creation, which relies heavily on the advertiser and is full of promotional artifacts [14]. Santos et al also recognize that user reviews are less technical, more readable, and more emotional compared to expert critics of video games [7]. In this way, the review content can be relatable to major audiences.…”

“…Typically, reviews of video games can be divided into two types based on the information source, namely user-generated reviews and influencer reviews [5][6][7]. Compared to ordinary users, influencers are perceived to be more trustworthy and credible because they are experts and experienced in the specific area, and have a high level of source credibility [8][9].…”

Mining Insights From Esports Game Reviews With an Aspect-Based Sentiment Analysis Framework

“…For instance, Huang et al [30] used LDA to identify various themes in Yelp reviews of restaurants, including service, value, decoration, and health. Feuerriegel et al [31] analyzed the impact of topics in corporate press releases on stock market returns, whereas Santos et al [32] applied LDA to assess disparities in video game evaluations by experts and amateurs.…”

A network-based matching design for text mining of hyper-polarised online reviews

“…The third section is an application of the methods. 1, 552, 750 public reviews are collected by 40, 665 items of theplatform Metacritic (Kasper et al 2019;Santos et al 2019). 218 EBI are detected and matched.…”