A Comparative Review of Skill Assessment: Performance, Prediction and Profiling

Abstract

Abstract. The assessment of chess players is both an increasingly attractive opportunity and an unfortunate necessity. The chess community needs to limit potential reputational damage by inhibiting cheating and unjustified accusations of cheating: there has been a recent rise in both. A number of counter-intuitive discoveries have been made by benchmarking the intrinsic merit of players' moves: these call for further investigation. Is Capablanca actually, objectively the most accurate World Champion? Has ELO rating inflation not taken place? Stimulated by FIDE/ACP, we revisit the fundamentals of the subject to advance a framework suitable for improved standards of computational experiment and more precise results. Other games and domains look to chess as demonstrator of good practice, including the rating of professionals making high-value decisions under pressure, personnel evaluation by Multichoice Assessment and the organization of crowd-sourcing in citizen science projects. The '3P' themes of performance, prediction and profiling pervade all these domains.

On the Causes and Consequences of Deviations from Rational Behavior

“…Data from chess tournaments have also been used to analyze various other research questions. These include, in particular, gender differences in patience (Gerdes et al, 2011), gender effects in competitiveness (Backus et al, 2016), gender and attractiveness (Dreber et al, 2013), self-selection and productivity in tournaments (Bertoni et al, 2015;Linnemmer and Visser, 2016), consequences of political ideology (Frank and Krabel, 2013), collusion (Moul and Nye, 2009), cheating (Barnes and Hernandez-Castro, 2015;Haworth et al, 2015) and indoor air quality (Künn et al, 2019). Recent work used chess data to compare the relative performance and strength of chess players in different time periods (Guid and Bratko, 2011;Alliot, 2017).…”