The Impact of Rating Scales on User’s Rating Behavior

Gena, Cristina; Brogi, Roberto; Cena, Federica; Vernero, Fabiana

doi:10.1007/978-3-642-22362-4_11

Cited by 19 publications

(29 citation statements)

References 9 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Explicit feedback recommenders let users rate items, (e.g. youtube.com, see also McNee et al 2002;Cena et al 2010;Gena et al 2011), critique items (see also Chen and Pu 2012;Viappiani et al 2006), assign weights to item attributes (see also Häubl et al 2004), or indicate their specific needs (e.g. HP.com 'help me choose', see also Felix et al 2001).…”

Section: Introductionmentioning

confidence: 99%

Explaining the user experience of recommender systems

Knijnenburg

Willemsen

Gantner

et al. 2012

User Model User-Adap Inter

570

340

View full text Add to dashboard Cite

Research on recommender systems typically focuses on the accuracy of prediction algorithms. Because accuracy only partially constitutes the user experience of a recommender system, this paper proposes a framework that takes a user-centric approach to recommender system evaluation. The framework links objective system aspects to objective user behavior through a series of perceptual and evaluative constructs (called subjective system aspects and experience, respectively). Furthermore, it incorporates the influence of personal and situational characteristics on the user experience. This paper reviews how current literature maps to the framework and identifies several gaps in existing work. Consequently, the framework is validated with four field trials and two controlled experiments and analyzed using Structural Equation Modeling. The results of these studies show that subjective system aspects and experience variables are invaluable in explaining why and how the user experience of recommender systems comes about. In all studies we observe that perceptions of recommendation quality and/or variety are important mediators in predicting the effects of objective system aspects on the three components of user experience: process (e.g. perceived effort, difficulty), system (e.g. perceived system effectiveness) and outcome (e.g. choice satisfaction). Furthermore, we find that these subjective aspects have strong and sometimes interesting behavioral correlates (e.g. reduced browsing indicates higher system effectiveness). They also show several tradeoffs between system aspects and personal and situational characteristics (e.g. the amount of preference feedback users provide is a tradeoff between perceived system usefulness and privacy concerns). These results, as well as the validated framework itself, provide a platform for future research on the user-centric evaluation of recommender systems.

show abstract

Section: Introductionmentioning

confidence: 99%

Explaining the user experience of recommender systems

Knijnenburg

Willemsen

Gantner

et al. 2012

User Model User-Adap Inter

570

340

View full text Add to dashboard Cite

show abstract

“…Apart from the ability to watch the trailer of a movie, users can not watch movies on the IMDb site before rating them, something possible in a system like Netflix. Moreover, the rating scale may also affect the users [Gena et al 2011], since the IMDb platform allows to rate movies using a 10-star feedback system, instead of the more common 5-star feedback system (like MovieLens).…”

Section: External Validitymentioning

confidence: 99%

A Framework for Dataset Benchmarking and Its Application to a New Movie Rating Dataset

Dooms

Bellogín

Pessemier

et al. 2016

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

Rating datasets are of paramount importance in recommender systems research. They serve as input for recommendation algorithms, as simulation data, or for evaluation purposes. In the past, public accessible rating datasets were not abundantly available, leaving researchers no choice but to work with old and static datasets like MovieLens and Netflix. More recently, however, emerging trends as social media and smartphones are found to provide rich data sources which can be turned into valuable research datasets. While dataset availability is growing, a structured way for introducing and comparing new datasets is currently still lacking. In this work, we propose a five-step framework to introduce and benchmark new datasets in the recommender systems domain. We illustrate our framework on a new movie rating dataset -called MovieTweetings -collected from Twitter. Following our framework, we detail the origin of the dataset, provide basic descriptive statistics, investigate external validity, report the results of a number of reproducible benchmarks, and conclude by discussing some interesting advantages and appropriate research use cases.

show abstract

“…Aiming at defining a general model of rating scales, we examined thirteen rating scales described by van Barneveld & van Setten (2004), Gena et al (2011) and Nobarany et al (2012): 3-, 5-and 10-point stars, bare numbers, smileys, sliders ranging -10/+10, -1/+1 and 0/10, likert-like scales ranging -10/+10 and 1/5 and 1-, 2-, and 3-point thumbs (see Figure 1). First, we identified a list of features which could be used to describe them, based on the literature and our insights, and we organized it in a model (Section 2.1).…”

Section: A Model Of Rating Scalesmentioning

confidence: 99%

“…Finally, Gena et al (Cena et al, 2010;Gena et al, 2011) adopted a different perspective, studying the influence of rating scales on user ratings. They introduced the concept of "rating scale personality" (resulting from user perception of a scale), and showed that it can influence …”

Section: Yesmentioning

confidence: 99%