Analysing Multimodal Texts in Science—a Social Semiotic Perspective

Abstract

Teaching and learning in science disciplines are dependent on multimodal communication. Earlier research implies that students may be challenged when trying to interpret and use different semiotic resources. There have been calls for extensive frameworks that enable analysis of multimodal texts in science education. In this study, we combine analytical tools deriving from social semiotics, including systemic functional linguistics (SFL), where the ideational, interpersonal, and textual metafunctions are central. In regard to other modes than writing—and to analyse how textual resources are combined—we build on aspects highlighted in research on multimodality. The aim of this study is to uncover how such a framework can provide researchers and teachers with insights into the ways in which various aspects of the content in multimodal texts are communicated through different semiotic resources. Furthermore, we aim to explore how different text resources interact and, finally, how the students, or authors of teaching resources, position themselves in relation to the subject. Data consist of one student text and one teaching resource text, both comprising drawn and written elements in combination with symbols. Our analyses of the student text suggest that the proposed framework can provide insights into students’ content knowledge and, hence, how construction of multimodal texts may be a useful tool for formative assessment. When it comes to teaching resources, the framework may be a useful tool for teachers when choosing resources, particularly in relation to students’ possibilities of meaning making when engaging with such texts, but also, as a basis for classroom discussions.

Primary Pupils’ Multimodal Representations in Worksheets—Text Work in Science Education

“…In this study, the worksheets as well as the interaction in the teaching practice were studied with general tools from SFL [25,30,[32][33][34]. To give an overview of the context in which the worksheets were used, the situational context is described in regard to the three register variables mentioned above, namely field (content in focus), tenor (the relationships between participants), and mode (the resources used in the interaction).…”

Meaning-Making in Ecology Education: Analysis of Students’ Multimodal Texts

“…To undertake a systematic analysis of student texts, we utilised a framework (see [49]), based on Systemic Functional Linguistics, SFL [41]. The framework applies aspects of Systemic Functional Grammar, SFG, combined with several SFL applications for analysing visual representations [17,50] and relations between written and drawn elements [51].…”

“…Earlier research in science education has predominantly focused on the ideational metafunction [31,51,53,54] whereas we base our analyses on all three metafunctions. The analytical framework is presented in detail in Wanselin, Danielsson, and Wikman [49]. Specific aspects of the framework employed in this study are given in Table 1 and they are presented in the following sections.…”

“…'disciplinary literacy', e.g., [72]). Such discussions can also act as a basis for formative teaching [73] (these ideas were further developed and discussed in [49]).…”

A Multimodal Text Analysis of K-Pop Fans Representation on Indonesia Film Television

“…A study on multimodal can explain how multimodal texts are communicated through several modes (Wanselin, Danielsson, & Wikman, 2021). Besides digital multimodal text, including film, animation, and video, two other types of multimodal text can be studied, namely print-based and live multimodal text.…”