Event Construal in Text and Image. Gaze Behavior and Discourse Interpretations

Abstract

The study explores the combinatorial prevalence effect in Event construal techniques in text and image components of heterosemiotic book pages. We hypothesize that their activity and contingency affect their interpretation, here tested in the oculographic experiment and discourse responses check. To proceed, we develop the parametric system applied for 100 book pages annotation and further statistical analysis. This study reveals the relevance of Truth, Type, Relation, Manageability, Completeness, Instantness, Achievement, Evaluation, Space location, Time location, Repeatability, Cause and effect parameter groups in Event construal in text and image as well as their resonance in concomitant activity. To select the samples serving as stimuli in the oculographic experiment, we apply Principal component analysis, which assigns Uniqueness indices to the samples, here ranging from 0.111 to 0.675, and provides diversity of Event construal techniques to be tested in terms of their interpretation. The results evidence that participants applied different text and image attention distribution patterns with longer fixations on text component in case the image displayed physical contact, static and desirable events. When the creation or destruction events, events-achievement, events located in time or causal events were not present in the text, the participants were more likely to address the image, not the text. Parameter activity also affects the choice of Descriptive, Narrative and Speculative discourse responses, with a restricted number of parameters stimulating Narrative discourse, with a restricted in text and vast in image number of parameters stimulating Speculative discourse, which evidences in favor of their more predetermined and predicted character. Hopefully, the results may be used to predict the interpretation effects and to further cognitive linguistic and semiotic research coordination.

Text and metatext event in the gaze behavior of impulsive and reflective readers

“…Still, the most important results were obtained in perspective construal. Text and metatext have displayed significant difference in subjectivity, which proves the observations made in (Kubryakova, Petrova, 2010;Iriskhanova, 2013;Rzheshevskaya, 2014;Kiose, 2022), but also attests to the importance of this relatively less studied aspect of event construal (in contrast with referent and event frame construal) in cognitive semantics. Overall, the semantic analysis proved the specificity of metatext construal (Kubryakova, Aleksandrova, 2008); additionally, it manifested the efficiency of contrastive study of text vs metatext, which might pave the way for further semantic and experimental research.…”

“…These three groups allow describe any event in any semiotic system, be it either text or image. The developed Event Construal Protocol has already been tested in multiple experimental studies (Kiose, Rzheshevskaya, 2021;Kiose et al, 2022) Centrality, Event Centrality (for further details see Kiose, 2022). However, to be incorporated into the pilot corpus MultiCORText, a trial version of protocol was developed applicable to texts and metatexts, in which Referents, Event Frame, Perspective within a discursive act are considered within linguistic representations.…”

“…-my happiness]), we annotated perspective in both; therefore, both subjectivation and intersubjectivation could be present in one unit. This perspective subgroup was the only one selected from a wider range of subgroups (see above) defined in (Verhagen, 2007;Iriskhanova, 2013;Rzheshevskaya, 2014;Kiose, 2022) due to low Cohen's Kappa received in agreement tests with other subgroups.…”