Optimal inference of sameness

Abstract

Deciding whether a set of objects are the same or different is a cornerstone of perception and cognition. Surprisingly, no principled quantitative model of sameness judgment exists. We tested whether human sameness judgment under sensory noise can be modeled as a form of probabilistically optimal inference. An optimal observer would compare the reliability-weighted variance of the sensory measurements with a set size-dependent criterion. We conducted two experiments, in which we varied set size and individual stimulus reliabilities. We found that the optimal-observer model accurately describes human behavior, outperforms plausible alternatives in a rigorous model comparison, and accounts for three key findings in the animal cognition literature. Our results provide a normative footing for the study of sameness judgment and indicate that the notion of perception as nearoptimal inference extends to abstract relations.Bayesian inference | ideal observer | decision making | vision A ccording to William James, the "sense of sameness is the very keel and backbone of our thinking" (1). Judging whether a set of stimuli are all the same (in one or multiple features) indeed seems to be a fundamental component of perception and cognition. When segmenting a visual scene, we make use of the fact that features such as orientation and color tend to be the same within an object and different between objects (2). Learning to categorize objects requires evaluating the aspects in which they are the same or different. Sameness judgment is also thought to play a role in the development of the abstract notion of equivalence, fundamental to learning mathematics (3). The central role of sameness judgment in perception and cognition is reflected in its prevalence in many psychophysical tasks, including matching to sample (4), change detection (5, 6), oddity search (7,8), and causal inference (9).The ability to judge sameness seems to have a long evolutionary history. In addition to humans, honey bees (10), pigeons (11), parrots (12), dolphins (13), Old World monkeys (5), New World monkeys (14,15), and apes (6, 7) can all learn to report whether pairs of objects are the same or different. This ability extends to larger groups of objects in both humans and nonhuman animals (16-21). It has been suggested that human capacity to recognize sameness at abstract levels is closely linked to the evolution of prefrontal cortex (22).In animal cognition experiments on sameness judgment, several intriguing trends have not yet been fully explained. First, when pigeons are trained to discriminate arrays of identical objects from arrays of different objects, their probability of reporting "different" is found to gradually increase with the amount of variability in the array (23, 24). For example, when an array contained four subsets of four icons each, with icons being identical within a subset, subjects reported "different" more frequently than when two subsets of eight icons were shown. Second, sameness judgment becomes easier with increasing number o...

Trial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization

“…Despite its simplicity, the task contains both ambiguity and variable sensory uncertainty-essential elements of natural forms of categorization. Evidence that organisms possess and use trial-to-trial knowledge of sensory uncertainty is abundant in perception (18,36,37), but rare in tasks with nontrivial stimulus categories (38,39). It might be worth studying the effects of sensory uncertainty on decision boundaries in more complex or more cognitive forms of categorization.…”

Bayesian statistics: relevant for the brain?

“…14]. Other studies construct Bayesian models of how subjects estimate the structure of the world [15–18]. A thriving community tries to relate actual behavior to predictions of optimal Bayesian behavior.…”

Visual Decisions in the Presence of Measurement and Stimulus Correlations

“…These models often focus on various decision rules that can be applied to the measurements. By contrast, the measurements themselves are usually modeled in a rather stereotypical fashion namely, as independent and normally distributed (e.g., Peterson, Birdsall, & Fox, 1954; Nolte & Jaarsma, 1967; Pelli, 1985; Graham, Kramer, & Yager, 1987; Palmer, Ames, & Lindsey, 1993; Baldassi & Burr, 2000; Baldassi & Verghese, 2002; van den Berg, Vogel, Josić, & Ma, 2012; Ma, Navalpakkam, Beck, van den Berg, & Pouget, 2011; Mazyar, van den Berg, & Ma, 2012). Both the assumption of independence and the assumption of gaussianity can be questioned.…”