Intelligent analysis of a visual scene requires that important regions be prioritized and attentionally selected for preferential processing. What is the basis for this selection? Here we compared the influence of meaning and image salience on attentional guidance in real-world scenes during two free-viewing scene description tasks. Meaning was represented by meaning maps capturing the spatial distribution of semantic features. Image salience was represented by saliency maps capturing the spatial distribution of image features. Both types of maps were coded in a format that could be directly compared to maps of the spatial distribution of attention derived from viewers’ eye fixations in the scene description tasks. The results showed that both meaning and salience predicted the spatial distribution of attention in these tasks, but that when the correlation between meaning and salience was statistically controlled, only meaning accounted for unique variance in attention. The results support theories in which cognitive relevance plays the dominant functional role in controlling human attentional guidance in scenes. The results also have practical implications for current artificial intelligence approaches to labeling real-world images.

Perception of a complex visual scene requires that important regions be prioritized and attentionally selected for processing. What is the basis for this selection? Although much research has focused on image salience as an important factor guiding attention, relatively little work has focused on semantic salience. To address this imbalance, we have recently developed a new method for measuring, representing, and evaluating the role of meaning in scenes. In this method, the spatial distribution of semantic features in a scene is represented as a meaning map. Meaning maps are generated from crowd-sourced responses given by naïve subjects who rate the meaningfulness of a large number of scene patches drawn from each scene. Meaning maps are coded in the same format as traditional image saliency maps, and therefore both types of maps can be directly evaluated against each other and against maps of the spatial distribution of attention derived from viewers’ eye fixations. In this review we describe our work focusing on comparing the influences of meaning and image salience on attentional guidance in real-world scenes across a variety of viewing tasks that we have investigated, including memorization, aesthetic judgment, scene description, and saliency search and judgment. Overall, we have found that both meaning and salience predict the spatial distribution of attention in a scene, but that when the correlation between meaning and salience is statistically controlled, only meaning uniquely accounts for variance in attention.

The world is visually complex, yet we can efficiently describe it by extracting the information that is most relevant to convey. How do the properties of real-world scenes help us decide where to look and what to say? Image salience has been the dominant explanation for what drives visual attention and production as we describe displays, but new evidence shows scene meaning predicts attention better than image salience. Here we investigated the relevance of one aspect of meaning, graspability (the grasping interactions objects in the scene afford), given that affordances have been implicated in both visual and linguistic processing. We quantified image salience, meaning, and graspability for real-world scenes. In three eyetracking experiments, native English speakers described possible actions that could be carried out in a scene. We hypothesized that graspability would preferentially guide attention due to its task-relevance. In two experiments using stimuli from a previous study, meaning explained visual attention better than graspability or salience did, and graspability explained attention better than salience. In a third experiment we quantified image salience, meaning, graspability, and reach-weighted graspability for scenes that depicted reachable spaces containing graspable objects.Graspability and meaning explained attention equally well in the third experiment, and both explained attention better than salience. We conclude that speakers use object graspability to allocate attention to plan descriptions when scenes depict graspable objects within reach, and otherwise rely more on general meaning. The results shed light on what aspects of meaning guide attention during scene viewing in language production tasks.

Torres EB, Isenhower RW, Yanovich P, Rehrig G, Stigler K, Nurnberger J, José JV. Strategies to develop putative biomarkers to characterize the female phenotype with autism spectrum disorders. J Neurophysiol 110: 1646 -1662, 2013. First published July 17, 2013 doi:10.1152/jn.00059.2013.-Current observational inventories used to diagnose autism spectrum disorders (ASD) apply similar criteria to females and males alike, despite developmental differences between the sexes. Recent work investigating the chronology of diagnosis in ASD has raised the concern that females run the risk of receiving a delayed diagnosis, potentially missing a window of opportunity for early intervention. Here, we retake this issue in the context of the objective measurements of natural behaviors that involve decisionmaking processes. Within this context, we quantified movement variability in typically developing (TD) individuals and those diagnosed with ASD across different ages. We extracted the latencies of the decision movements and velocity-dependent parameters as the hand movements unfolded for two movement segments within the reach: movements intended toward the target and withdrawing movements that spontaneously, without instruction, occurred incidentally. The stochastic signatures of the movement decision latencies and the percent of time to maximum speed differed between males and females with ASD. This feature was also observed in the empirically estimated probability distributions of the maximum speed values, independent of limb size. Females with ASD showed different dispersion than males with ASD. The distinctions found for females with ASD were better appreciated compared with those of TD females. In light of these results, behavioral assessment of autistic traits in females should be performed relative to TD females to increase the chance of detection.

No abstract