Visual and auditory cues of assertions and questions in brazilian portuguese and Mexican Spanishy

Abstract

The aim of this paper is to compare the multimodal production of questions in two different language varieties: Brazilian Portuguese and Mexican Spanish. Descriptions of the auditory and visual cues of two speech acts, assertions and questions, are presented based on Brazilian and Mexican corpora. The sentence “Como você sabe” was produced as an yes-no (echo) question and an assertion by ten speakers (five male) from Rio de Janeiro and the sentence “Apaga la tele” was produced as a yes-no question and an assertion by five speakers (three male) from Mexico City. The results show that, whereas the Brazilian Portuguese and Mexican Spanish assertions are produced with different F0 contours and different facial expressions, questions in both languages are produced with specific F0 contours but similar facial expressions. The outcome of this comparative study suggests that lowering the eyebrows, tightening the lid and wrinkling the nose can be considered question markers in both language varieties.

Visual channel facilitates the comprehension of the intonation of Brazilian Portuguese wh-questions and wh-exclamations: evidence from congruent and incongruent stimuli

“…Since Miranda et al (2019) concluded that both channels are integrated during speech perception of BP wh-question and wh-exclamations with listeners benefiting from the presentation of stimuli with congruent auditory and visual cues, in this paper, the research question and hypothesis of this study are: Will the mismatched facial and audio cues allow the recognition of the speech acts? We hypothesize, based on previous studies, that the incongruent stimuli will hinder the identification of the pragmatic meaning expressed in the auditory modality by intonation, although the comprehension of the speech act will still be possible.…”

“…In the last few decades, a substantial number of multimodal studies analyzed several functions expressed by intonation, such as demarcation (phrasing the speech stream into smaller units), highlighting (the place of prominence within an utterance) and the distinction of sentence mode (statements, questions, requests, commands etc.). For instance, regarding yes-no/echo questions and assertions, due to their essential function as building blocks in conversation (Holler et al, 2018), these speech acts are vastly explored in a multimodal approach in several languages such as Swedish (House, 2002), American English (Srinivasan & Massaro, 2003), British English (Fisher, 1969), Catalan (Borràs-Comes & Prieto, 2011), Mexican Spanish (Gomes da Silva, 2019; Miranda et al, 2020a), Brazilian Portuguese (henceforth BP) (Miranda et al, 2021;Peres et al, 2011) and European Portuguese (Cruz et al, 2017). Overall, these studies support that different facial expressions participate in the transmission of pragmatic meaning conveyed by intonation and that visual cues are perceptually integrated with auditory cues, meaning that listeners benefit from prosodic functions presented in a bimodal condition.…”

“…The author found out that the more expressive the information was, the more use of facial gestures in the production of speakers. In Miranda et al (2019), 10 BP speakers were filmed and recorded while producing the sentence 'Como você sabe' as wh-questions (meaning, 'How do you know?') and wh-exclamations (meaning, 'How clever you are!').…”

“…The present study aims to expand the multimodal analysis of Miranda et al (2019) by verifying the relative strength of the auditory and visual cues in the perception of Brazilian Portuguese wh-questions and wh-exclamations with a new approach. Stimuli constructed based on mismatched modalities were used in this study's perceptual experiment design, since very little is known about the contribution of facial cues to the perception of BP wh-questions and wh-exclamations.…”

“…Although previous research has indicated that specific facial signals are associated with questions and assertions, few works have targeted whquestions and wh-exclamations. Miranda et al (2019) described specific facial gestures in the production of BP wh-questions and wh-exclamations. For whquestions, a lowered eyebrow plus the head turned right were the most typical combination of gestures, whereas for wh-exclamations, raised eyebrow, lips corner raiser and up and down head movement were found, as seen in Figure 2.…”