Quality and Leniency in Online Collaborative Rating Systems

Abstract

The emerging trend of social information processing has resulted in Web users' increased reliance on usergenerated content contributed by others for information searching and decision making. Rating scores, a form of user-generated content contributed by reviewers in online rating systems, allow users to leverage others' opinions in the evaluation of objects. In this article, we focus on the problem of summarizing the rating scores given to an object into an overall score that reflects the object's quality. We observe that the existing approaches for summarizing scores largely ignores the effect of reviewers exercising different standards in assigning scores. Instead of treating all reviewers as equals, our approach models the leniency of reviewers, which refers to the tendency of a reviewer to assign higher scores than other coreviewers. Our approach is underlined by two insights: (1) The leniency of a reviewer depends not only on how the reviewer rates objects, but also on how other reviewers rate those objects and (2) The leniency of a reviewer and the quality of rated objects are mutually dependent. We develop the leniency-aware quality, or LQ model, which solves leniency and quality simultaneously. We introduce both an exact and a ranked solution to the model. Experiments on real-life and synthetic datasets show that LQ is more effective than comparable approaches. LQ is also shown to perform consistently better under different parameter settings.

Online Reputation Model Using Moving Window

“…In each iteration the MAE is calculated, and the overall MAE is averaged at the end. The reputation models that have been used for comparison purposes are: Naïve average, LQ [9], Dirichlet [7], and Fuzzy model [3]. As discussed in literature, there are two approaches to find the appropriate size of the window: fixed size and duration.…”

“…User reliability means that his provided ratings are very close to the global agreement for all products he rated. Likewise, Lauw et al [9] studied the leniency ad strictness of users in providing ratings. Lenient user are those who frequently provide positive ratings regardless the actual product quality.…”

Comparative Analysis of Online Rating Systems

“…In this study we used eight state-of-art models are: Mean, Median, BetaDR [1], Bayesian [6], Dirichlet [5], IMDb, Fuzzy rating [2] and LQ [7]. For comparison purpose we used 10-Fold cross validation.…”

“…They defined a measure to calculate that closeness and use it with their weight average model. Lauw et al [7] studied the leniency of user while they rate products. They proposed a function that can calculate the leniency and strictness of user and reflect that as weight.…”

A rating aggregation method for generating product reputations

“…On the other hand, Riggs and Wilensky [6] performed collaborative quality filtering, based on the principle of finding the most reliable users. Lauw et al introduced the Leniency-Aware Quality (LQ) Model [7] which is a weighted average model that uses users' ratings tendency as weights. Using fuzzy models are also popular in calculating reputation scores because fuzzy logic provides rules for reasoning with fuzzy measures [8,9].…”