Understanding environmental sounds in sentence context

Abstract

There is debate about how individuals use context to successfully predict and recognize words. One view argues that context supports neural predictions that make use of the speech motor system, whereas other views argue for a sensory or conceptual level of prediction. While environmental sounds can convey clear referential meaning, they are not linguistic signals, and are thus neither produced with the vocal tract nor typically encountered in sentence context. We compared the effect of spoken sentence context on recognition and comprehension of spoken words versus nonspeech, environmental sounds. In Experiment 1, sentence context decreased the amount of signal needed for recognition of spoken words and environmental sounds in similar fashion. In Experiment 2, listeners judged sentence meaning in both high and low contextually constraining sentence frames, when the final word was present or replaced with a matching environmental sound. Results showed that sentence constraint affected decision time similarly for speech and nonspeech, such that high constraint sentences (i.e., frame plus completion) were processed faster than low constraint sentences for speech and nonspeech. Linguistic context facilitates the recognition and understanding of nonspeech sounds in much the same way as for spoken words. This argues against a simple form of a speech-motor explanation of predictive coding in spoken language understanding, and suggests support for conceptual-level predictions.

Cross-Modal Interaction Between Auditory and Visual Input Impacts Memory Retrieval

“…For instance, our finding that spatial memory was enhanced in response to congruent environmental sounds (but not congruent words or neutral/incongruent sounds and words) could conceivably be attributed to greater identifiability of sound vs. word stimuli. Given that prior research indicates that environmental sounds are typically recognized at comparable, or even lower rates relative to spoken words ( Uddin et al, 2018 ; Bartolotti et al, 2020 ), however, the advantage for congruent environmental sounds observed in the present study is unlikely to be attributable to greater recognition of sounds vs. words. We note that the identifiability of items within a given block (words and sounds) is unlikely to account for effects of condition (congruent, incongruent, and neutral), as lists were counterbalanced across participants so that each item was presented in each of the conditions (i.e., as the congruent and incongruent auditory stimulus, as well as the visual target).…”

Contextual constraints on the activation of lexical forms by nonlinguistic sounds.

“…With respect to linguistic constraints, not only did Van Petten and Rheinfelder (1995) find that environmental sound primes influenced the processing of semantically related spoken word targets but spoken word primes also influenced the processing of semantically related environmental sound targets. Uddin et al (2018) also found that high cloze probability sentence contexts (e.g., “He bought diapers for his …”) facilitated responses to both corresponding spoken words (e.g., “baby”) and environmental sounds (e.g., a sound produced by a baby; see also Uddin et al, 2018). These findings reveal that like words, the processing of environmental sounds is affected by the immediate context.…”

Naming pictures and sounds: Stimulus type affects semantic context effects.

“…For example, perceptual features like furry, four-footed, or barks may either be connected directly to the word dog or indirectly via the concept DOG. Although to date our knowledge about how environmental sounds are processed and named is still limited (Hocking et al, 2013; Uddin et al, 2018), it is reasonable to assume that in the context of current models environmental sounds activate one or more perceptual features in a similar way as pictures do and that these features then (directly or indirectly) activate abstract lexical representations. Still, there are also differences between picture and sound processing that might affect task performance.…”