Allen Newell's Program of Research: The Video‐Game Test

Abstract

Newell (1973) argued that progress in psychology was slow because research focused on experiments trying to answer binary questions, such as serial versus parallel processing. In addition, not enough attention was paid to the strategies used by participants, and there was a lack of theories implemented as computer models offering sufficient precision for being tested rigorously. He proposed a three-headed research program: to develop computational models able to carry out the task they aimed to explain; to study one complex task in detail, such as chess; and to build computational models that can account for multiple tasks. This article assesses the extent to which the papers in this issue advance Newell's program. While half of the papers devote much attention to strategies, several papers still average across them, a capital sin according to Newell. The three courses of action he proposed were not popular in these papers: Only two papers used computational models, with no model being both able to carry out the task and to account for human data; there was no systematic analysis of a specific video game; and no paper proposed a computational model accounting for human data in several tasks. It is concluded that, while they use sophisticated methods of analysis and discuss interesting results, overall these papers contribute only little to Newell's program of research. In this respect, they reflect the current state of psychology and cognitive science. This is a shame, as Newell's ideas might help address the current crisis of lack of replication and fraud in psychology.

Unsupervised identification of internal perceptual states influencing psychomotor performance

“…Relatedly, and partly as a consequence of this limitation, studies of video games that involve repeated measures often aggregate data within and across sessions of engagement. Previous work has advised against this on theoretical [11,12,13] as well as empirical grounds, demonstrating how certain insights into individual differences [21] or skill acquisition [68,69] can only be achieved after disaggregating data and considering behaviour in a more detailed fashion. Although this study involved detailed analysis of behaviour through time, we are guilty of the sin of aggregation as we too considered our sample as a single homogenous group, despite variation in players' average scores indicating a heterogeneity in skill level.…”

“…This makes the analysis of performance using total scores difficult, as they may mask underlying factors that can vary across trials and sessions, for instance, changes in player behaviour as a response to novel situations in the game. A proposed solution is that researchers look past total scores by interrogating variables describing components of performance, such as patterns of players' control inputs and decisions [11,12,13]. This is a particular advantage afforded by digital games as experimental tasks, as their programming often allows researchers to extract high-density behavioural data describing multiple aspects of performance.…”

“…This makes the analysis of performance using total scores difficult, as they may mask underlying factors that can vary across trials and sessions, for instance, changes in player behaviour as a response to novel situations in the game. A proposed solution is that researchers look past total scores by interrogating variables describing components of performance, such as patterns of players' control inputs and decisions [11,12,13].…”

Maximizing the Potential of Digital Games for Understanding Skill Acquisition

“…We gather observations on how learning occurs, but real progress will come with testing theories of the cognitive mechanisms which allow individuals to acquire skills. Gobet (2017) makes the case that theoretical progress in this area will require computational accounts of complete task performance. While we can't hope to even sketch such a comprehensive theory here, we believe that learning curve analysis -including formal modeling of individual learning curves -is a necessary step and will allow research on digital games to contribute to the wider topic of skill development.…”

Maximizing the potential of digital games for understanding skill acquisition

“…We gather observations on how learning occurs, but real progress will come with testing theories of the cognitive mechanisms which allow individuals to acquire skills. Gobet (2017) makes the case that theoretical progress in this area will require computational accounts of complete task performance. While we can't hope to even sketch such a comprehensive theory here, we do wish to highlight some theoretical and methodological challenges which will need to be overcome on the way to such a theory.…”