Category systems for real-world scenes

Anderson, Matt D; Graf, Erich W.; Elder, James H.; Ehinger, Krista A.; Adams, Wendy J.

doi:10.1167/jov.21.2.8

Cited by 7 publications

(22 citation statements)

References 86 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been argued that global scene information extracted using the GIST descriptor predicts human ratings of spatial envelope properties, and that these are in turn informative of scene category (Oliva & Torralba, 2001). However, our findings imply that, when stimuli are reduced to the very rudimentary information that is extracted by the GIST descriptor, human categorization performance is in fact far from perfect, making the GIST descriptor alone an unlikely candidate for the effortless categorization ability of humans (see also Anderson et al, 2021). Furthermore, we use stimuli extracting information from 6 × 6 windows as compared to the 4 × 4 windows typically used with the GIST descriptor.…”

Section: Discussionmentioning

confidence: 81%

What makes a scene? Fast scene categorization as a function of global scene information at different resolutions.

Wiesmann¹,

Võ²

2022

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

The ability to quickly and accurately categorize our environment into meaningful scene categories has been linked to the fast processing of global scene properties. Here, we causally tested this claim by reducing scene images to different sets of global scene properties with varying degrees of spatial resolution. In Experiment 1, human observers reached above-chance categorization accuracy for most stimulus conditions, indicating that images reduced to global scene properties indeed allow for correct categorization. In Experiment 2, we demonstrated that these features can be extracted from only 30 ms of stimulus presentation. However, accuracy in both experiments was far below performance on unmanipulated images and differed strongly between conditions. Images with higher resolutions yielded better performance than those with lower resolutions. In Experiment 3, we related this performance advantage to the identification of single objects in stimuli with higher resolutions which supports the notion that object information is used for scene categorization. Taken together, we show that global scene properties are useful but not sufficient for fast scene categorization. Instead, localized information is crucial in this process, presumably because it conveys object identities. Public Significance StatementWe can categorize images of different environments within the glimpse of an eye. In this study, we show that, while global, unlocalized information is sufficient for above-chance categorization performance, the localization of visual information within an image is crucial for reaching the full efficiency of this task.

show abstract

Section: Discussionmentioning

confidence: 81%

What makes a scene? Fast scene categorization as a function of global scene information at different resolutions.

Wiesmann¹,

Võ²

2022

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

show abstract

“…e behavioural gesture presented in each frame of the video sequence is regarded as an independent behavioural state, and each independent state is combined using probability so that in the state space method, the different motion sequences can be regarded as a continuous behaviour consisting of a sequence of independent states [17]. e probabilistic likelihood of each state is compared, with the standard sequence of behaviours, and after the complete behaviours have been compared, the overall likelihood value of the whole behaviour is calculated and the standard behaviour with the highest overall likelihood value, as shown in Figure 2.…”

Section: A Gaussian High-dimensional Random Matrix Design For Student...mentioning

confidence: 99%

A Gaussian High-Dimensional Random Matrix-Based Method for Detecting Abnormal Student Behaviour in Chinese Language Classrooms

Zhang

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

In this paper, a Gaussian high-dimensional random matrix approach is used to conduct an in-depth study and analysis of the detection of abnormal student behaviour in Chinese language classrooms, and a corresponding model is designed for practical application. The Gaussian high-dimensional random matrix technique and recurrent neural network technique are applied to build a basic technical framework for intelligent proctoring. An innovative method of human posture estimation is used to complete the proctoring task, and the structure of the Gaussian high-dimensional random matrix method is simplified and the model is compressed to achieve real-time and parallelism of the method. A PolSAR sparse representation classification (OGRM_SRC) algorithm based on orthogonal Gaussian random matrices (OGRM) is proposed to construct an observation dictionary based on polarisation features and OGRM to select typical ground target samples in polarized feature images. The system not only functions as a psychometric assessment of students but also is capable of detecting and analysing abnormal student behaviour based on the improved forest algorithm. The residuals of the pixel to be classified are calculated relative to each atom in the observation dictionary, and the minimum reconstruction residual is used as the classification criterion. The OGRM_SRC algorithm is applied to classify the pixel, and the final classification results are obtained and evaluated for accuracy. We propose a fine-grained action recognition optimisation method for recurrent neural networks, fusing temporal attention information and spatial attention information to improve the action recognition method and improve the model’s ability to recognise fine-grained abnormal behaviours such as looking left, looking right, and copying with head down. Based on this, we conducted some comparative experiments to validate the effectiveness of our work and verified that the system achieved a recognition efficiency of 76.8 FPS and a recognition accuracy of 98.5% in a graphical computing processor.

show abstract

“…This Chapter is published as a research article: Anderson, M. D., Graf, E. W., Elder, J. H., Ehinger, K. A., & Adams, W. J. (2021).…”

Section: B Publication Notementioning

confidence: 99%

“…The assumption that semantic categorization is primarily driven by global image features also has limited support. GIST features are unreliable predictors of spatial structure properties across different image databases (Anderson et al, 2021). Moreover, objects are processed just as quickly as entire scenes (Fabre-Thorpe, 2011; Joubert et al, 2007;Rousselet et al, 2005;VanRullen & Thorpe, 2001), and scene categorization is impaired when embedded objects are incongruent with the scene (e.g., a man-made object in a natural scene, Davenport, 2007;Davenport & Potter, 2004;Joubert et al, 2007;Mack & Palmeri, 2010).…”

Section: Semantic Categorymentioning

confidence: 99%

See 1 more Smart Citation

Category and Depth Discrimination in Real-World Scenes

Anderson¹

2022

Preprint

View full text Add to dashboard Cite

Visual understanding of real-world scenes is near-instantaneous. Humans can extracta wealth of information, including spatial structure, semantic category, and theidentity of embedded objects, from images viewed for fewer than 100 msecs. Visualprocessing has capacity limits, and, as a result, the computational processes thatunderlie this behaviour must be highly efficient. Computational theories of realworldscene perception model early image processing in various ways. In Chapter 1,I review these theories, and in Chapter 2, I review the role of depth cues in rapidvisual processing. This discussion reveals three problems: (i) Tests of the agreementbetween model predictions and human responses may be biased by the arbitrarychoice of category system, (ii) Current models posit that scene semantics isestimated from spatial structure properties, but empirical support for this position isinconsistent, and (iii) The time-course of depth estimation in real-world scenes ispoorly understood. To address these problems, three empirical papers are presentedin Chapters 3, 4, and 5. In Chapter 3, I propose and validate a novel clusteringalgorithm that can be applied to image databases to derive category systems forvisual experiments. In Chapters 3 and 4, I examine the relationship between spatialstructure and semantic information, and find little support for the position that spatialstructure properties inform semantic discrimination. In Chapters 4 and 5, Icharacterize the time-course of depth processing for images presented for <267msecs, and conclude that binocular disparity and elevation cues contribute to realworldperception shortly after image onset (<50 msecs). These findings are discussedtogether in Chapter 6. This thesis contributes to the evaluation of modern models ofreal-world scene perception, and helps to characterize how visual understandingunfolds over time.

show abstract

Category systems for real-world scenes

Cited by 7 publications

References 86 publications

What makes a scene? Fast scene categorization as a function of global scene information at different resolutions.

What makes a scene? Fast scene categorization as a function of global scene information at different resolutions.

A Gaussian High-Dimensional Random Matrix-Based Method for Detecting Abnormal Student Behaviour in Chinese Language Classrooms

Category and Depth Discrimination in Real-World Scenes

Contact Info

Product

Resources

About