The tug of war between phonological, semantic and shape information in language-mediated visual search

Huettig, Falk; McQueen, James M.

doi:10.1016/j.jml.2007.02.001

Cited by 274 publications

(495 citation statements)

References 42 publications

Supporting

Mentioning

421

Contrasting

Order By: Relevance

“…1. When presented with scenes containing phonological onset competitors and semantic competitors, high literates looked first at phonological competitors and then later at semantic competitors once information within the unfolding speech mismatched with the name of the phonological competitor, replicating earlier research (Huettig & McQueen, 2007). Low literates on the other hand only displayed increased fixation of semantic competitors, at no point fixating phonological competitors consistently more than unrelated distractors.…”

Section: Explicit and Implicit Phonological Processing Taskssupporting

confidence: 74%

See 1 more Smart Citation

Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Smith

Monaghan

Huettig

2014

Cognitive Psychology

Self Cite

View full text Add to dashboard Cite

a b s t r a c tLearning to read and write requires an individual to connect additional orthographic representations to pre-existing mappings between phonological and semantic representations of words. Past empirical results suggest that the process of learning to read and write (at least in alphabetic languages) elicits changes in the language processing system, by either increasing the cognitive efficiency of mapping between representations associated with a word, or by changing the granularity of phonological processing of spoken language, or through a combination of both. Behavioural effects of literacy have typically been assessed in offline explicit tasks that have addressed only phonological processing. However, a recent eye tracking study compared high and low literate participants on effects of phonology and semantics in processing measured implicitly using eye movements. High literates' eye movements were more affected by phonological overlap in online speech than low literates, with only subtle differences observed in semantics. We determined whether these effects were due to cognitive efficiency and/or granularity of speech processing in a multimodal model of speech processing -the amodal shared resource model (ASR, Smith, Monaghan, & Huettig, 2013a,b). We found that cognitive efficiency in the model had only a marginal effect on semantic processing and did not affect performance for phonological processing, whereas fine-grained versus coarse-grained

show abstract

Section: Explicit and Implicit Phonological Processing Taskssupporting

confidence: 74%

“…Huettig & McQueen, 2007). This was done by splitting the remainder of the test trial, post preview, equally into two further windows, an early window (time step 8-18) and a late window (time step 19-29).…”

Section: Simulating the Effects Of Grain Size And Cognitive Efficiencmentioning

confidence: 99%

Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Smith

Monaghan

Huettig

2014

Cognitive Psychology

Self Cite

View full text Add to dashboard Cite

show abstract

“…Previous experiments on spoken-word recognition with the visual world paradigm used pictures of objects or shapes (e.g., Allopenna et al, 1998;Dahan et al, 2001) or printed words (Huettig & McQueen, 2007;McQueen & Viebahn, 2007). We used both pictures and words here and combined them in a novel way.…”

Section: Discussionmentioning

confidence: 99%

Processing reduced word-forms in speech perception using probabilistic knowledge about speech production.

Mitterer¹,

McQueen²

2009

Journal of Experimental Psychology: Human Perception and Perfor

Self Cite

View full text Add to dashboard Cite

Two experiments examined how Dutch listeners deal with the effects of connected-speech processes, specifically those arising from word-final /t/ reduction (e.g., whether Dutch [tas] is tas, bag, or a reduced-/t/ version of tast, touch). Eye movements of Dutch participants were tracked as they looked at arrays containing 4 printed words, each associated with a geometrical shape. Minimal pairs (e.g., tas/tast) were either both above (boven) or both next to (naast) different shapes. Spoken instructions (e.g., "Klik op het woordje tas boven de ster," [Click on the word bag above the star]) thus became unambiguous only on their final words. Prior to disambiguation, listeners' fixations were drawn to /t/-final words more when boven than when naast followed the ambiguous sequences. This behavior reflects Dutch speechproduction data: /t/ is reduced more before /b/ than before /n/. We thus argue that probabilistic knowledge about the effect of following context in speech production is used prelexically in perception to help resolve lexical ambiguities caused by continuous-speech processes.Keywords: spoken-word recognition, continuous-speech processes, /t/-reduction, eye tracking, DutchContinuous speech is peppered with noncanonical pronunciations of words. Continuous-speech processes, such as assimilation, reduction, and deletion, operate during speech production and can substantially change the way words are realized. For example, a speaker of British English may say [waImbɔʔ1] in a casual request for a wine bottle, but say [waIn] and [bɔtəl] when asked how these words should be pronounced. It is the listener's task to recognize spoken words in spite of this variability. A particularly acute problem is when a continuous-speech process creates a lexical ambiguity. We examine that situation here, focusing on the effects of word-final /t/ reduction. If a speaker does not clearly pronounce the final [t] of duct, for example, how does the listener work out that they are not talking about a duck?Two answers to this question have been proposed. The first is based on lexical storage. There are in fact three radically different versions of this view. Lahiri and Marslen-Wilson (1991) and Lahiri and Reetz (2002) proposed a model of the mental lexicon in which entries are highly abstract and phonologically underspecified. A lexical entry for a given word only includes the phonological features that are reliably associated with tokens of that word. For instance, the alveolar nasal consonants in English are assumed to be specified only as nasal, with no specification of place of articulation, which may vary due to phonological assimilation. The token [waIm] can be recognized as an instance of the word wine because there is no mismatch between the last nasal segment [m] in the input and the lexical representation for this segment. Similarly, lexical entries may be underspecified for the presence of a word-final /t/. The input [d∧k] would thus match a stored lexical representation corresponding to duct. The second lexical-storage a...

show abstract

“…The fixation proportions to the different objects on the screen were analyzed using two independent variables: word-learning condition (two levels: Auditory Forms Only and Auditory+Spelled Forms) and target vowel (two levels: /ae/ and /e/). The impact of these factors was evaluated for looks to the targets and competitor effects (comparing competitors and distractors) in subject and item analysis (see, e.g., Dahan & Tanenhaus, 2004;Huettig & McQueen, 2007). Note that word-learning condition is a betweensubjects variable in the subject analysis, but a within-subjects variable in the item analysis, while target vowel is a between-items and a within-subjects variable.…”

Section: Testing Phasementioning

confidence: 99%

Novel second-language words and asymmetric lexical access

Escudero

Hayes‐Harb

Mitterer

2008

Journal of Phonetics

196

272

View full text Add to dashboard Cite

The lexical and phonetic mapping of auditorily confusable L2 nonwords was examined by teaching L2 learners novel words and by later examining their word recognition using an eye-tracking paradigm. During word learning, two groups of highly proficient Dutch learners of English learned 20 English nonwords, of which 10 contained the English contrast /e/-ae/ (a confusable contrast for native Dutch speakers). One group of subjects learned the words by matching their auditory forms to pictured meanings, while a second group additionally saw the spelled forms of the words. We found that the group who received only auditory forms confused words containing /ae/ and /e/ symmetrically, i.e., both /ae/ and /e/ auditory tokens triggered looks to pictures containing both /ae/ and /e/. In contrast, the group who also had access to spelled forms showed the same asymmetric word recognition pattern found by previous studies, i.e., they only looked at pictures of words containing /e/ when presented with /e/ target tokens, but looked at pictures of words containing both /ae/ and /e/ when presented with /ae/ target tokens. The results demonstrate that L2 learners can form lexical contrasts for auditorily confusable novel L2 words. However, and most importantly, this study suggests that explicit information over the contrastive nature of two new sounds may be needed to build separate lexical representations for similar-sounding L2 words. r

show abstract

The tug of war between phonological, semantic and shape information in language-mediated visual search

Cited by 274 publications

References 42 publications

Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Literacy effects on language and vision: Emergent effects from an amodal shared resource (ASR) computational model

Processing reduced word-forms in speech perception using probabilistic knowledge about speech production.

Novel second-language words and asymmetric lexical access

Contact Info

Product

Resources

About