No abstract

As we act on the world around us, our eyes seek out objects we plan to interact with. A growing body of evidence suggests that overt visual attention selects objects in the environment that could be interacted with, even when the task precludes physical interaction. Our previous work showed objects that afford grasping interactions influenced attention when static scenes depicted reachable spaces, and attention was otherwise better explained by general meaning (Rehrig, Peacock, et al., 2021). Because grasping is but one of many object interactions, our previous work may have downplayed the influence of object affordances on attention. The current study investigated the relationship between overt visual attention and object affordances versus broadly construed semantic information in scenes as speakers describe possible actions. In addition to meaning and grasp maps—which capture informativeness and grasping object affordances in scenes, respectively—we introduce interact maps, which capture affordances more broadly. In a mixed-effects analysis of 3 eyetracking experiments, interact map values predicted fixated regions in all experiments, whereas there was no main effect of meaning, and grasp maps marginally predicted fixated locations for scenes that depicted reachable spaces only. Our findings suggest speakers consistently allocate attention to scene regions that could be readily interacted with when describing the possible actions in a scene, while the other variants of semantic information tested (graspability and general meaning) have a compensatory or additive influence on attention. The current study clarifies the importance of object affordances in guiding visual attention in scenes.

Many characters in written Chinese incorporate components (radicals) that provide cues to meaning. These cues are often partial, and some are misleading because they are unrelated to the character's meaning. Previous studies have shown that radicals influence the reader's processing of the characters in which they occur (e.g., Feldman & Siok, 1999). We investigated whether readers automatically activate the semantic information associated with a radical even when it is irrelevant to the character's meaning, using a modified version of the Van Orden (1987) task with auditory semantic relatedness ratings on test items. Fifty-one Mandarin speakers participated in the study. On each trial they saw a reference category such as "animal" prior to seeing a character then indicated whether the target character was a member of that category.Decisions were slower and less accurate when a target that is not a member of the target category contained a radical that is. For example, if the category is "found in the kitchen," the answer for the target 券 "ticket" is no; however the character contains the misleading radical 刀 "knife". These patterns suggest that readers process the semantics of the radical even when it is not relevant to the meaning of the character. The results further verify the role of radical semantics in character processing and raise questions as to whether repetitions of experience with the idiosyncrasies of the script may result in some of the irrelevant semantics influencing the meaning of the character.

This work investigates the linearization strategies used by speakers when describing real-world scenes to better understand production plans for multi-utterance sequences. Scene meaning predicts visual attention across tasks, but does not predict the order in which objects in scenes are described, contrary to previous work suggesting a tight coupling between the visual attention and language production systems in simple description tasks. In this study, 30 participants described real-world scenes aloud. To investigate which semantic features of scenes predict order of mention, we quantified three features (meaning, graspability, and interactability) using two techniques (whole-object ratings and feature map values). We found that object-level semantic features, broadly defined and affordance-based, predicted order of mention in a scene description task. Our findings provide the first evidence for an object-related semantic feature that guides linguistic ordering decisions and offer theoretical support for the role of object semantics in scene viewing and description.

Many characters in written Chinese incorporate components (radicals) that provide cues to meaning. The cues are often partial, and some are misleading because they are unrelated to the character’s meaning. Previous studies have shown that radicals influence the processing of the characters in which they occur (e.g., Feldman & Siok, 1999). We investigated whether readers automatically activate the semantics associated with a radical even when it is irrelevant to the character’s meaning, using a modified version of the Van Orden (1987) task. Fifty-one Mandarin speakers participated in the study. On each trial they were shown a reference category such as “animal” prior to seeing a target character then indicated whether the target character was a member of that category. Decisions were slower and less accurate when a target that is not a member of the target category contained a radical that is. For example, if the category is “found in the kitchen,” the answer for the target 券 (ticket) is no; however the character contains the misleading radical 刀 (knife). These patterns suggest that readers process the semantics of the radical even when it is not relevant to the meaning of the character. The results present challenges for theories in which whole characters are the units of processing in reading Chinese. They also raise questions as to whether repetitions of this experience may result in some of the irrelevant semantics influencing the meaning of the character.

No abstract