Evaluation of Wine Judge Performance through Three Characteristics: Bias, Discrimination, and Variation

Stokes, Lynne

doi:10.1017/s1931436100001413

Cited by 28 publications

(28 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, reliable assessments of wine quality require judges to discriminate among wines of varying quality on a consistent basis. While past work by the authors and others (Cao and Stokes, 2010) quantified judge inconsistency, this work provides a foundation for establishing reasonable standards for judge performance. Unfortunately, when applied to a recognized group of wine experts, most fail to meet these standards.…”

Section: Discussionmentioning

confidence: 95%

Criteria for Accrediting Expert Wine Judges

Hodgson

2013

J Wine Econ

Self Cite

View full text Add to dashboard Cite

A test for evaluating wine judge performance is developed. The test is based on the premise that an expert wine judge will award similar scores to an identical wine. The definition of "similar" is parameterized to include varying numbers of adjacent awards on an ordinal scale, from No Award to Gold. For each index of similarity, a probability distribution is developed to determine the likelihood that a judge might pass the test by chance alone. When the test is applied to the results from a major wine competition, few judges pass the test. Of greater interest is that many judges who fail the test have vast professional experience in the wine industry. This leads to us to question the basic premise that experts are able to provide consistent evaluations in wine competitions and, hence, that wine competitions do not provide reliable recommendations of wine quality. (JEL Classifications: C02, C12, D81)

show abstract

Section: Discussionmentioning

confidence: 95%

Criteria for Accrediting Expert Wine Judges

Hodgson

2013

J Wine Econ

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many research efforts indicate that wine judges may demonstrate intra-and interinconsistencies while tasting designated wines [33][34][35][36][37][38][39][40]. The Wine Spectator is a wine magazine company that provides consistent creditable wine reviews periodically by a group of wine region specific reviewers.…”

Section: Wine Spectatormentioning

confidence: 99%

Wineinformatics: Using the Full Power of the Computational Wine Wheel to Understand 21st Century Bordeaux Wines from the Reviews

2021

View full text Add to dashboard Cite

Although wine has been produced for several thousands of years, the ancient beverage has remained popular and even more affordable in modern times. Among all wine making regions, Bordeaux, France is probably one of the most prestigious wine areas in history. Since hundreds of wines are produced from Bordeaux each year, humans are not likely to be able to examine all wines across multiple vintages to define the characteristics of outstanding 21st century Bordeaux wines. Wineinformatics is a newly proposed data science research with an application domain in wine to process a large amount of wine data through the computer. The goal of this paper is to build a high-quality computational model on wine reviews processed by the full power of the Computational Wine Wheel to understand 21st century Bordeaux wines. On top of 985 binary-attributes generated from the Computational Wine Wheel in our previous research, we try to add additional attributes by utilizing a CATEGORY and SUBCATEGORY for an additional 14 and 34 continuous-attributes to be included in the All Bordeaux (14,349 wine) and the 1855 Bordeaux datasets (1359 wines). We believe successfully merging the original binary-attributes and the new continuous-attributes can provide more insights for Naïve Bayes and Supported Vector Machine (SVM) to build the model for a wine grade category prediction. The experimental results suggest that, for the All Bordeaux dataset, with the additional 14 attributes retrieved from CATEGORY, the Naïve Bayes classification algorithm was able to outperform the existing research results by increasing accuracy by 2.15%, precision by 8.72%, and the F-score by 1.48%. For the 1855 Bordeaux dataset, with the additional attributes retrieved from the CATEGORY and SUBCATEGORY, the SVM classification algorithm was able to outperform the existing research results by increasing accuracy by 5%, precision by 2.85%, recall by 5.56%, and the F-score by 4.07%. The improvements demonstrated in the research show that attributes retrieved from the CATEGORY and SUBCATEGORY has the power to provide more information to classifiers for superior model generation. The model build in this research can better distinguish outstanding and class 21st century Bordeaux wines. This paper provides new directions in Wineinformatics for technical research in data science, such as regression, multi-target, classification and domain specific research, including wine region terroir analysis, wine quality prediction, and weather impact examination.

show abstract

“…In the next section, we describe a method that provides still more information about the individual differences in the judge's score characteristics. Cao and Stokes (2010) proposed a Bayesian ordinal model to evaluate wine judge performance by identifying which of the three judge characteristics-bias, discrimination ability, and random variation-are responsible for disagreement among judges. Judge bias measures the systematic difference between a judge's score and the average score of all judges.…”

Section: B a Parametric Analysis: Product-moment Correlationsmentioning

confidence: 99%

“…For more details on the prior assignment of the parameters and Bayesian computation, see Cao and Stokes (2010).…”

Section: A Parametric Analysis: Bayesian Ordinal Modelmentioning

confidence: 99%

Analyses of Wine-Tasting Data: A Tutorial

Olkin

Lou²,

Stokes

2014

J Wine Econ

Self Cite

View full text Add to dashboard Cite

The purpose of this paper is to provide a tutorial of data analysis methods for answering questions that arise in analyzing data from wine-tasting events: (i) measuring agreement of two judges and its extension to m judges; (ii) making comparisons of judges across years; (iii) comparing two wines; (iv) designing tasting procedures to reduce burden of multiple tastings;(v) ranking of judges; and (vi) assessing causes of disagreement. In each case we describe one or more analyses and make recommendations on the conditions of use for each. (JEL Classifications: C10, C12, C13, C59, C90)

show abstract

Evaluation of Wine Judge Performance through Three Characteristics: Bias, Discrimination, and Variation

Cited by 28 publications

References 12 publications

Criteria for Accrediting Expert Wine Judges

Criteria for Accrediting Expert Wine Judges

Wineinformatics: Using the Full Power of the Computational Wine Wheel to Understand 21st Century Bordeaux Wines from the Reviews

Analyses of Wine-Tasting Data: A Tutorial

Contact Info

Product

Resources

About