Evaluating Wine-Tasting Results and Randomness with a Mixture of Rank Preference Models

Bodington, Jeffrey C.

doi:10.1017/jwe.2014.41

Cited by 19 publications

(22 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Equation 2expresses L MLE using a Plackett-Luce rank preference model of each taster's scores. Among many applications, Plackett-Luce was employed to evaluate taste test results for sushi by Chen (2014), animal feed by Marden (1995) and wine by Bodington (2015aBodington ( , 2015b. Plackett-Luce employs a preference probability for each wine (r i for wine i taster t and totals of W wines and T tasters) that expresses the chance that the wine is most-preferred among the alternatives.…”

Section: Description Of the Wop Challengementioning

confidence: 99%

“…None of the 1328 wines in the 2016 Challenge were assigned the same score by all of the six or seven of the judges who evaluated each wine. Cao (2014) and Bodington (2012Bodington ( , 2015aBodington ( , 2015b posited that observed scores are a mixture of consensus, idiosyncratic and random expressions of preference. Numerous evaluations showed that there is often a consensus among some judges that some wines are better than others.…”

Section: Dispersion Of Scores and Potential Randomness In Awardsmentioning

confidence: 99%

See 1 more Smart Citation

The 2016 Wines of Portugal Challenge: general implications of more than 8400 wine-score observations

Bodington

Malfeito‐Ferreira

2017

Journal of Wine Research

View full text Add to dashboard Cite

The Wines of Portugal Challenge is an annual competition among wines produced by over 1000 vintners in over 30 of the country's wine growing regions. In 2016, judges assigned scores to over 1300 wines resulting in over 8400 wine-score observations. Analysis of that large sample yields implications about wine judges' ratings that are difficult to detect with statistical significance in the small samples that are typical of most wine tastings. The Challenge's frequency distribution of scores showed left skewness and local peaks just below the score thresholds for bronze, silver and gold awards. Student's t-tests showed that there were no significant differences in scores assigned by gender-of-judge, nationality-of-judge and to wines from different regions. However, judges did assign higher scores to sweet wines than to other types of wine. While the dispersion in scores was material, p-values showed that the aggregate order of rating was very unlikely to be random and the distributions of mean scores showed that the strengths of judges' preferences against the least-preferred wines were stronger than those in favor of the most-preferred wines. Ties between wines' mean scores were common and could be broken by several methods including the preference probabilities implied by a Plackett-Luce model.

show abstract

Section: Description Of the Wop Challengementioning

confidence: 99%

Section: Dispersion Of Scores and Potential Randomness In Awardsmentioning

confidence: 99%

The 2016 Wines of Portugal Challenge: general implications of more than 8400 wine-score observations

Bodington

Malfeito‐Ferreira

2017

Journal of Wine Research

View full text Add to dashboard Cite

show abstract

“…are required for formal statistical answers according to the Journal of Wine Economics [1]. In the past decade, many researchers focused on these problems with small to medium sized wine datasets [2][3][4][5]. However, to the best of our knowledge, no research is being performed on analyzing the consistency of wine judges with a large-scale dataset.…”

Section: Introductionmentioning

confidence: 99%

Wineinformatics: A Quantitative Analysis of Wine Reviewers

et al. 2018

View full text Add to dashboard Cite

Data Science is a successful study that incorporates varying techniques and theories from distinct fields including Mathematics, Computer Science, Economics, Business and domain knowledge. Among all components in data science, domain knowledge is the key to create high quality data products by data scientists. Wineinformatics is a new data science application that uses wine as the domain knowledge and incorporates data science and wine related datasets, including physicochemical laboratory data and wine reviews. This paper produces a brand-new dataset that contains more than 100,000 wine reviews made available by the Computational Wine Wheel. This dataset is then used to quantitatively evaluate the consistency of the Wine Spectator and all of its major reviewers through both white-box and black-box classification algorithms. Wine Spectator reviewers receive more than 87% accuracy when evaluated with the SVM method. This result supports Wine Spectator’s prestigious standing in the wine industry.

show abstract

“…Cao (2014) applied a mixture model with random-ranking and consensus-ranking components to the results of the 2009 California State Fair Commercial Wine Competition. Bodington (2015) applied a mixture of rank-preference models to the ranks assigned by experienced tasters during a blind tasting of Pinot Gris, and the results implied that common-preference agreement among tasters exceeded the random expectation of illusory agreement.…”

Section: Introductionmentioning

confidence: 99%

“…The mixture of Plackett-Luce models applied to ranked data for a tasting of Pinot Gris in Bodington (2015) is summarized in Section II. Transforming numerical scores into ranks and choice axioms are discussed in Section III, and an addition to the mixture model to handle ties between numerical scores appears in Section IV.…”

Section: Introductionmentioning

confidence: 99%

Testing a Mixture of Rank Preference Models on Judges' Scores in Paris and Princeton

Bodington¹

2015

J Wine Econ

Self Cite

View full text Add to dashboard Cite

Rank preference and mixture models have been employed to evaluate the ranks assigned by consumers in taste tests of beans, cheese, crackers, salad dressings, soft drinks, sushi, animal feed, and wine. In many wine tastings, including the famous 1976 Judgment of Paris and the 2012 Judgment of Princeton, judges assign scores rather than ranks, and those scores often include ties. This article advances the application of ranking and mixture models to wine-tasting results by modifying the established use of a Plackett-Luce rank preference model to accommodate scores and ties. The modified model is tested and then employed to evaluate the Paris and Princeton wine-tasting results. Test results show that the mixture model is an accurate predictor of observed rank densities. Results for Paris and Princeton show that the group preference orders implied by the mixture model are highly correlated with the orders implied by widely employed rank-sum methods. However, the mixture model satisfies choice axioms that rank-sum methods do not, it yields an estimate of the proportion of scores that appear to be assigned randomly, and it also yields a preference order based on nonrandom preferences that tasters appear to hold in common. (JEL Classifications: A10, C00, C10, C12, D12)

show abstract

Evaluating Wine-Tasting Results and Randomness with a Mixture of Rank Preference Models

Cited by 19 publications

References 20 publications

The 2016 Wines of Portugal Challenge: general implications of more than 8400 wine-score observations

The 2016 Wines of Portugal Challenge: general implications of more than 8400 wine-score observations

Wineinformatics: A Quantitative Analysis of Wine Reviewers

Testing a Mixture of Rank Preference Models on Judges' Scores in Paris and Princeton

Contact Info

Product

Resources

About