Minimal theory of 3D vision: new approach to visual scale and visual shape

Abstract

Since Kepler and Descartes in the early-1600s, vision science has been committed to a triangulation model of stereo vision. But in the early-1800s, we realized that disparities are responsible for stereo vision. And we have spent the past 200 years trying to shoe-horn disparities back into the triangulation account. The first part of this article argues that this is a mistake, and that stereo vision is a solution to a different problem: the eradication of rivalry between the two retinal images, rather than the triangulation of objects in space. This leads to a ‘minimal theory of 3D vision’, where 3D vision is no longer tied to estimating the scale, shape, and direction of objects in the world. The second part of this article then asks whether the other aspects of 3D vision, which go beyond stereo vision, really operate at the same level of visual experience as stereo vision? I argue they do not. Whilst we want a theory of real-world 3D vision, the literature risks giving us a theory of picture perception instead. And I argue for a two-stage theory, where our purely internal ‘minimal’ 3D percept (from stereo vision) is linked to the world through cognition. This article is part of a discussion meeting issue ‘New approaches to 3D vision’.

Toward a Theory of Perspective Perception in Pictures

“…This echoes theories from real-world 3D vision that separate shape and scale, whether in pictures or in real-world 3D vision (Vishwanath, 2014;Linton, 2022), as well as evidence of dissociation between shape and location estimation (Loomis et al, 2002). Vishwanath (2014) points out that internal 3D structure (what I am calling "shape") in real-world vision can be determined solely from relative depth estimates, but absolute depth estimates require additional cues.…”

“…The hypotheses here combine and extend several currently-separate sets of ideas: the roles of fixations, foveal vision, and vision-at-a-glance (Rensink, 2000;Rosenholtz, 2020;Wolfe et al, 2022); effective formulations for removing perspective distortions in computational photography, summarized by the Direct View Condition (Zorin and Barr, 1995); the fragmentary nature of 3D vision (Koenderink et al, 2008;Linton, 2022); and Local Slant Compensation (Vishwanath et al, 2005).…”

“…Many conventional theories of vision-as well as common-sense notions of it-assume that we viewers see the visible space in front of our eyes, understand it, and build a mental 3D model of it, e.g., (Linton, 2022). However, in the past few decades, many surprising experimental results challenged this view.…”

“…Classical notions of visual space-a person's visual experience and internal representation of 3D spacetreat it as a coherent geometric representation of the real world (Suppes, 1977;Erkelens, 2015). However, many studies identify inconsistencies and distortions in visual space that cannot be explained by any metric geometry (Linton, 2022): distortions of estimated depth due to irrelevant factors (Johnston, 1991;Todd and Norman, 2003;Campagnoli et al, 2022;Vishwanath, 2014), inconsistencies between estimates of depth, slant, curvature, and/or shape (Di Luca et al, 2010;Koenderink, 1998;Loomis et al, 2002;Koenderink et al, 2002), and internal inconsistency of multiple relative judgements (Koenderink et al, 2008;Svarverud et al, 2012;Vuong et al, 2019). Some recent theories explain inconsistent results by treating visual space as fragmentary in some way, with, for example, separate representations for different surfaces (Koenderink, 1998), different distance ranges (Vishwanath, 2022), or for action versus perception (Goodale and Milner, 1992).…”

From pictures to reality: modelling the phenomenology and psychophysics of 3D perception

“… 7 Simultaneous localization and mapping (SLAM). A method in computer vision where coordinates of points making up objects and surfaces in a scene are derived from multiple 2D pictorial images of the scene (see the Introduction to this issue) [ 62 ]. …”