Two perils of binary categorization: Why the study of concepts can’t afford true/false testing

Abstract

In this opinion piece, we outline two shortcomings in experimental design that limit the claims that can be made about concept learning in animals. On the one hand, most studies of concept learning train too few concepts in parallel to support general claims about their capacity of subsequent abstraction. On the other hand, even studies that train many categories of stimulus in parallel only test one or two stimuli at a time, allowing even a simplistic learning rule to succeed by making informed guesses. To demonstrate these shortcomings, we include simulations performed using an off-the-shelf image classifier.These simulations demonstrate that, when either training or testing are overly simplistic, a classification algorithm that is incapable of abstraction nevertheless yields levels of performance that have been described in the literature as proof of concept learning in animals.PeerJ PrePrints | http://dx.doi.org/10.7287/peerj.preprints.688v2 | CC-BY 4.0 Open Access | rec: 4

Inferential Learning of Serial Order of Perceptual Categories by Rhesus Monkeys (Macaca mulatta)

“…For example, pictures of flowers never included eyes, the absence of which could be used as a cue for that category. The present study does not rule out the possibility that subjects relied on a classifier that was tailor-made for the stimulus set, shaped by this study’s specific feedback (Jensen & Altschul, 2015). However, this does not alter our conclusions regarding serial learning.…”

“…Traditionally, studies of categorization in animals initially train category membership using the match-to-sample paradigm (Herrnstein 1985, Crouzet et al 2012), a match-to-stimulus paradigm (Fabre-Thorpe 1998, Basile & Hampton 2013), or a match-to-category design (Freedman and Miller 2001). In these paradigms, subjects evaluate stimuli one at a time, a process that is highly vulnerable to a “guessing” strategy (Jensen & Altschul 2015). The categorical TI experiment is distinct from these procedures because it required subjects to evaluate two categories from ten possible pairings.…”