Enhancing writing analytics in science education research with machine learning and natural language processing—Formative assessment of science and non-science preservice teachers’ written reflections

Abstract

IntroductionScience educators use writing assignments to assess competencies and facilitate learning processes such as conceptual understanding or reflective thinking. Writing assignments are typically scored with holistic, summative coding rubrics. This, however, is not very responsive to the more fine-grained features of text composition and represented knowledge in texts, which might be more relevant for adaptive guidance and writing-to-learn interventions. In this study we examine potentials of machine learning (ML) in combination with natural language processing (NLP) to provide means for analytic, formative assessment of written reflections in science teacher education.MethodsML and NLP are used to filter higher-level reasoning sentences in physics and non-physics teachers’ written reflections on a standardized teaching vignette. We particularly probe to what extent a previously trained ML model can facilitate the filtering, and to what extent further fine-tuning of the previously trained ML model can enhance performance. The filtered sentences are then clustered with ML and NLP to identify themes and represented knowledge in the teachers’ written reflections.ResultsResults indicate that ML and NLP can be used to filter higher-level reasoning elements in physics and non-physics preservice teachers’ written reflections. Furthermore, the applied clustering approach yields specific topics in the written reflections that indicate quality differences in physics and non-physics preservice teachers’ texts.DiscussionOverall, we argue that ML and NLP can enhance writing analytics in science education. For example, previously trained ML models can be utilized in further research to filter higher-level reasoning sentences, and thus provide science education researchers efficient mean to answer derived research questions.

Empowering Scientific Literacy of Science Teachers: Systematic Literature Review (SLR)

“…Based on the research design graph shown in Figure 3, mixed methods were most widely used in the articles, namely as many as 14 articles (Alghamdi et al, 2022;Jimenez et al, 2022;Busch et al, 2023;Lilly et al, 2023;Luft et al, 2022;Peters-Burton et al, 2023;Wulff et al, 2023;Fridberg et al, 2023;Davis & Palincsar, 2023;Nilsson & Lund, 2023;Mouza et al, 2023;Abdulbakioglu et al, 2022;Buma & Sibanda 2022;Becerra et al, 2023). Apart from that, quantitative descriptive also has eight articles (Poce et al, 2019;Schofield et al, 2023;Schofield et al, 2023;Rachmatullah et al, 2023;Zoupidis et al, 2022;Vasconcelos & Paz, 2023;Nja et al, 2022;& Alghamdi 2023).…”

Physics language and language use in physics—What do we know and how AI might enhance language-related research and instruction

“…The performance and generalizability could be substantially boosted by using LLMs to transform the input data [161]. Moreover, the LLM-based model could undergo additional fine-tuning for various contexts, allowing for the classification of written reflections from non-physics students [165]. This offers new possibilities in PER to share models and thus scale research.…”

Human-Centric AI : Enhancing User Experience through Natural Language Interfaces

“…By looking at the various levels of these AI kinds, it offers a thorough grasp of the technological underpinnings that allow the development of intelligent systems. (Wulff et al, 2023). The ability of Natural Language Interfaces (NLIs) to understand and interact with human language has made them an essential component in the creation of user-centric artificial intelligence.…”