Recurrent computations for visual pattern completion

Abstract

SignificanceThe ability to complete patterns and interpret partial information is a central property of intelligence. Deep convolutional network architectures have proved successful in labeling whole objects in images and capturing the initial 150 ms of processing along the ventral visual cortex. This study shows that human object recognition abilities remain robust when only small amounts of information are available due to heavy occlusion, but the performance of bottom-up computational models is impaired under limited visibility. The results provide combined behavioral, neurophysiological, and modeling insights showing how recurrent computations may help the brain solve the fundamental challenge of pattern completion.

Beyond the feedforward sweep: feedback computations in the visual cortex

“…It has been shown that, electrophysiologically, the initial sweep of rapid visually selective signals along the ventral VC (VVC) is unaffected by backward masking 14 . Consistent with the idea that backward masking interrupts recurrent processing, recent work has shown that the introduction of a rapid mask interferes with the ability to perform visual recognition tasks that require more processing time such as pattern completion, 48 as elaborated in the section entitled “Generalization in visual recognition” (Fig. 3A and 3B).…”

“…Modified from Ref. 48. (C) The fraction of neural response variance explained for neurons in macaque inferior temporal cortex.…”

“…The inadequacy of purely bottom‐up signals for pattern completion suggests that the ability to infer the whole from the parts relies on additional horizontal and/or top‐down signals. Indeed, computational work has shown that the addition of recurrent computations to DCNNs can help solve the problem of pattern completion 48,83 . Additionally, there is physiological evidence that strongly suggests that top‐down signals from the PFC onto the VVC play an important role during the recognition of occluded objects 83,88 .…”

“…2D). 48 Indeed, a related idea referred to as adaptive computing is gaining traction in computer vision and natural language processing and is being actively explored both with feedforward 49 and recurrent networks. 50,51 An experimental technique that has been used to impose rapid processing is backward masking.…”

“…Indeed, computational work has shown that the addition of recurrent computations to DCNNs can help solve the problem of pattern completion. 48,83 Additionally, there is physiological evidence that strongly suggests that top-down signals from the PFC onto the VVC play an important role during the recognition of occluded objects. 83,88 It is also known that familiar object shapes have an influence on image segmentation, 34,89,90 and it is possible that the ability to complete patterns and make inferences from partial information is enhanced by top-down effects on image segmentation.…”