Development of multimodal discourse comprehension: cohesive use of space by gestures

Sekiné, Kazuki; Kita, Sotaro

doi:10.1080/23273798.2015.1053814

Cited by 14 publications

(13 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This, in turn, helped children find and encode actor‐specific features, which led to better actor recognition memory. It is well‐documented that pointing gestures (e.g., Langton, O'Malley, & Bruce, ) and the deictic component of iconic gestures (i.e., location in gesture space, at which iconic gestures are produced; Sekine & Kita, ) can direct the recipient's attention to particular areas of the interactional space. The current study demonstrates for the first time that iconicity in iconic gesture can also direct the recipient's attention to a particular part of an event that includes the referent of the gesture.…”

Section: Discussionmentioning

confidence: 99%

Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events

Aussems

Kita

2017

Child Development

Self Cite

View full text Add to dashboard Cite

An experiment with 72 three-year-olds investigated whether encoding events while seeing iconic gestures boosts children's memory representation of these events. The events, shown in videos of actors moving in an unusual manner, were presented with either iconic gestures depicting how the actors performed these actions, interactive gestures, or no gesture. In a recognition memory task, children in the iconic gesture condition remembered actors and actions better than children in the control conditions. Iconic gestures were categorized based on how much of the actors was represented by the hands (feet, legs, or body). Only iconic hand-as-body gestures boosted actor memory. Thus, seeing iconic gestures while encoding events facilitates children's memory of those aspects of events that are schematically highlighted by gesture.

show abstract

Section: Discussionmentioning

confidence: 99%

Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events

Aussems

Kita

2017

Child Development

Self Cite

View full text Add to dashboard Cite

show abstract

“…Behaviorally, they start to comprehend iconic gestures representing entities at around 3 years of age ( Stanfield et al, 2014 ) and iconic gestures representing events at around 4 years of age ( Glasser et al, 2018 ). Studies showed that 3-year olds could not integrate speech and gesture, whereas 5-year old and adults did (e.g., Sekine and Kita, 2015 ; Sekine et al, 2015 ). Moreover, children starting from 6 years of age integrate speech and gesture in an online fashion comparable to adults ( Dick et al, 2012 ; Sekine et al, in press ).…”

Section: Individual Differences In Gesture Processingmentioning

confidence: 99%

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Özer

Göksun

2020

Front. Psychol.

View full text Add to dashboard Cite

Speakers use spontaneous hand gestures as they speak and think. These gestures serve many functions for speakers who produce them as well as for listeners who observe them. To date, studies in the gesture literature mostly focused on group-comparisons or the external sources of variation to examine when people use, process, and benefit from using and observing gestures. However, there are also internal sources of variation in gesture use and processing. People differ in how frequently they use gestures, how salient their gestures are, for what purposes they produce gestures, and how much they benefit from using and seeing gestures during comprehension and learning depending on their cognitive dispositions. This review addresses how individual differences in different cognitive skills relate to how people employ gestures in production and comprehension across different ages (from infancy through adulthood to healthy aging) from a functionalist perspective. We conclude that speakers and listeners can use gestures as a compensation tool during communication and thinking that interacts with individuals' cognitive dispositions.

show abstract

“…Previous studies have attributed their results to both dimensions. For instance, while Cassell et al (1999) attribute the effect in their study to handedness, suggesting that addressees associate each hand with a different referent, other studies assume location in space to be the determining factor (e.g., Goodrich Smith and Hudson Kam, 2012; Gunter et al, 2015; Sekine and Kita, 2015, 2017; Gunter and Weinbrenner, 2017). However, none of the studies provide decisive evidence either way.…”

Section: Introductionmentioning

confidence: 96%

“…Some studies suggest that localizing gestures that are spatially congruent with previous gestures can facilitate processing in comparison to spatially incongruent localizing gestures (Cassell et al, 1999), while others suggest that congruent localizing gestures facilitate processing in comparison to speech alone (Gunter et al, 2015; Gunter and Weinbrenner, 2017). Finally, anaphoric localizing gestures are shown to reinforce expectations about referent resolution in speech (Goodrich Smith and Hudson Kam, 2012; Nappa and Arnold, 2014), and to help identify referents (Sekine and Kita, 2015).…”

Section: Introductionmentioning

confidence: 99%

“…Fourth, there is potentially altered allocation of attention to gesture. Some studies have chosen to blur (Gunter et al, 2015; Gunter and Weinbrenner, 2017), or cover the narrators’ faces with masks (Sekine and Kita, 2015, 2017). While this technique might control important aspects of an experiment (e.g., being able to use the same audio for different videos), it also means that participants’ attention to the gestures may be increased.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Addressees Are Sensitive to the Presence of Gesture When Tracking a Single Referent in Discourse

2019

View full text Add to dashboard Cite

Production studies show that anaphoric reference is bimodal. Speakers can introduce a referent in speech by also using a localizing gesture, assigning a specific locus in space to it. Referring back to that referent, speakers then often accompany a spoken anaphor with a localizing anaphoric gesture (i.e., indicating the same locus). Speakers thus create visual anaphoricity in parallel to the anaphoric process in speech. In the current perception study, we examine whether addressees are sensitive to localizing anaphoric gestures and specifically to the (mis)match between recurrent use of space and spoken anaphora. The results of two reaction time experiments show that, when a single referent is gesturally tracked, addressees are sensitive to the presence of localizing gestures, but not to their spatial congruence. Addressees thus seem to integrate gestural information when processing bimodal anaphora, but their use of locational information in gestures is not obligatory in every discourse context.

show abstract

Development of multimodal discourse comprehension: cohesive use of space by gestures

Cited by 14 publications

References 31 publications

Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events

Seeing Iconic Gestures While Encoding Events Facilitates Children's Memory of These Events

Gesture Use and Processing: A Review on Individual Differences in Cognitive Resources

Addressees Are Sensitive to the Presence of Gesture When Tracking a Single Referent in Discourse

Contact Info

Product

Resources

About