Tuning the ensemble: Incidental skewing of the perceptual average through memory-driven selection.

Abstract

The process by which multiple items within an object grouping are rapidly summarized along a given visual dimension into a single mean value (i.e., perceptual averaging) has increasingly been shown to interact dynamically with visual working memory (VWM). Commonly, this interaction is studied with respect to the influence of perceptual averaging over VWM, but it is also the case that VWM can support perceptual averaging. Here, we argue that, in the presence of memory-matching elements, VWM exerts an obligatory influence over perceptual averaging even when it is detrimental to do so. Over four experiments, we tested our hypothesis by having individuals perform a mean orientation estimation task while concurrently maintaining a colored object in VWM. We anticipated that mean orientation reports would be attracted to the local mean of memory-matching items if such items are prioritized in perceptual average judgments. This was indeed the case as we observed a persistent bias in mean orientation judgments toward the subset mean of items matching the VWM item color, despite color being entirely irrelevant to the mean orientation task. Our results thus highlight a goal-invariant influence of VWM over perceptual averaging, which we attribute to amplification through memory-driven selection. Public Significance StatementCurrent understanding of the interaction between perceptual averaging and visual working memory (VWM) has largely centered on the influence of the former over the latter, with much less consideration of a potential bidirectional relationship between these two systems. In this study, we show that not only does VWM alter perceptual averaging judgments but that it also does so automatically (i.e., even when it is costly to do so). More broadly, this work provides confirmatory support to the idea that the amplification of items within object ensembles is set by an underlying priority map, which guides selective attention on the basis of physical salience, top-down goals, emotional valence, and reward history.

Memory matching features bias the ensemble perception of facial identity

“…Accordingly, in consideration of this hierarchical structure, matching features through the VWM task would lead to different effects in the highlevel ensemble perception. However, this prediction seems contradict to previous literature in which a similar amplification effect driven by physical saliences was evidenced in both low-level (e.g., orientations of lines; Williams et al, 2021) and high-level (e.g., facial expressions; Goldenberg et al, 2021;Goldenberg et al, 2022;Yang and Baek, 2022) ensemble coding. This suggests that the amplification effect is independent of ensemble perception levels.…”

“…Results of their study showed that the accuracy of the size averaging task was unchanged according to levels of working memory load (i.e., remembering zero, two, or four items), nor was it affected by the VWM task. Considering these findings together, we can conclude that the existence of an inter-task shared feature is crucial for the influence of a memorized item on averaging estimations for a group of properties (Epstein and Emmanouil, 2017;Williams et al, 2021). Overall, these findings aligned well with the amplification hypothesis of perceptual averaging (Kanaya et al, 2018), which stated that physically salient elements are involuntarily and automatically weighted more than less salient elements in the contribution of average estimations (Kanaya et al, 2018;Iakovlev and Utochkin, 2021).…”

“…All the aforementioned studies have discussed or confirmed the influence of memory matching features on low-level ensemble coding (Epstein and Emmanouil, 2017;Williams et al, 2021). However, few studies have focused on high-level ensemble coding.…”

“…Humans have developed a crucial ability called ensemble perception, in which a group of stimulus properties, ranging from low-level features (e.g., orientation, size, and color) to high-level properties (e.g., facial expression, face identity), are rapidly extracted to form summary statistics, such as a mean or a variance of stimuli ( Ariely, 2001 ; Chong and Treisman, 2005 ; Haberman and Whitney, 2012 ; Whitney and Yamanashi Leib, 2018 ). Past studies have shown that this ability has a mutual effect along with visual working memory (VWM) and have found that summary statistics interact with remembered items within the memory representation ( Brady and Alvarez, 2011 ; Bauer, 2017 ; Corbett, 2017 ; Corbin and Crawford, 2018 ; Utochkin and Brady, 2020 ; Williams et al, 2021 ). With respect to the impact of ensemble coding on items stored in VWM, discoveries are that estimates of the memorized item were readily biased toward the mean of a subset in the same color as this item ( Brady and Alvarez, 2011 ), or a subset in conformity with Gestalt principles ( Corbett, 2017 ), or a group of homogeneity of items ( Utochkin and Brady, 2020 ).…”

“…With respect to the impact of ensemble coding on items stored in VWM, discoveries are that estimates of the memorized item were readily biased toward the mean of a subset in the same color as this item ( Brady and Alvarez, 2011 ), or a subset in conformity with Gestalt principles ( Corbett, 2017 ), or a group of homogeneity of items ( Utochkin and Brady, 2020 ). Notably, recent work has focused on the influence of VWM on the process of ensemble coding ( Bauer, 2017 ; Epstein and Emmanouil, 2017 ; Dodgson and Raymond, 2020 ; Williams et al, 2021 ; Jia et al, 2022 ).…”