Comparing rapid scene categorization of aerial and terrestrial views: A new perspective on scene gist

Abstract

Scene gist, a viewer's holistic representation of a scene from a single eye fixation, has been extensively studied for terrestrial views, but not for aerial views. We compared rapid scene categorization of both views in three experiments to determine the degree to which diagnostic information is view dependent versus view independent.We found large differences in observers' ability to rapidly categorize aerial and terrestrial scene views, consistent with the idea that scene gist recognition is viewpoint dependent.In addition, computational modeling showed that training models on one view (aerial or terrestrial) led to poor performance on the other view, thereby providing further evidence of viewpoint dependence as a function of available information. Importantly, we found that rapid categorization of terrestrial views (but not aerial views) was strongly interfered with by image rotation, further suggesting that terrestrial-view scene gist recognition is viewpoint dependent, with aerial-view scene recognition being viewpoint independent. Furthermore, rotation-invariant texture images synthesized from aerial views of scenes were twice as recognizable as those synthesized from terrestrial views of scenes (which were at chance), providing further evidence that diagnostic information for rapid scene categorization of aerial views is viewpoint invariant. We discuss the results within a perceptual-expertise framework that distinguishes between configural and featural processing, where terrestrial views are more effectively processed due to their predictable view-dependent configurations whereas aerial views are processed less effectively due to reliance on view-independent features.

The role of edge-based and surface-based information in natural scene categorization: Evidence from behavior and event-related potentials

“…After the blank, six category names appeared on the screen from left to right, for which the locations corresponded to the keys D, F, G, H, J, and K on the keyboard. On each trial, the locations of the six category names were randomly assigned, and thus, the participants would not prepare their response before its appearance and no response bias toward a favored location would contaminate the results 3 (Loschky, Ringer, Ellis, & Hansen, 2015). The participants were forced to make a choice among the six categories by pressing the corresponding key, and there was no time limit for them to make the choice.…”

Seeing the World from Above: Training with Aerial Scenes Reveals the Effects of Experience on Scene Recognition

“…Further, since aerial scenes are processed less efficiently than terrestrial scenes (Ringer et al, 2018;Śikl et al, 2019), we hypothesized that categorization of aerial scenes at baseline would be poorer than that of terrestrial scenes. However, at the same time because of the participants' unfamiliarity with viewing scenes from an aerial perspective (Loschky et al, 2015) we predicted greater learning for aerial scenes compared to terrestrial scene learning.…”

“…Since humans are terrestrial animals, the visual system has adapted to efficiently encode a number of environmental constants, or diagnostic features, that reflect a ground-based viewpoint (Greene & Oliva, 2009) rather than an aerial one. An example of this is the detection of horizontal and vertical edges nested within the perceptual upright layout relative to the horizon (the gravitational frame; Loschky, Ringer, Ellis, & Hansen, 2015;Ringer & Loschky, 2018). These diagnostic features are absent or completely transformed in aerial scenes.…”

“…Experience with a particular scene across various positions will yield viewpointindependent representations that can produce fast and efficient processing akin to the formation of viewpoint-invariant object representations with increasing experience (Kravitz, Vinson, & Baker, 2008). Thus, in the present work we aimed to establish how people develop mental correspondences between two different viewpoints of the same type of scenes over the course of extensive training as means of assessing the formation of visual expertise in scene categorization (see also Loschky et al, 2015). The rationale for using two viewpoints of the same place was to determine the extent to which scene expertise involves an increase in the observer's ability to extract invariant information from one viewpoint and apply it to another.…”

Coupling ground-level panoramas and aerial imagery for change detection

“…In the aim of enhancing current results, we will now consider more advanced images comparison methods and will complete our preprocessing pipeline by other steps such as photometric correction. Comparing ground-based and aerial imagery is still a challenging issue, as noticed by a recent study from Loschky et al (2015). Other future works include enlarging geographic extent of the study area and increasing the volume of test data and metrics.…”