Improving summary writing through formative feedback in a technology‐enhanced learning environment

Abstract

Summary writing is a useful instructional tool for learning. However, summary writing is a challenge to many students. This mixed‐method study examined the potential of the Student Mental Model Analyzer for Research and Teaching (SMART) system to help students produce summaries that reflect key concepts and relations in a text. SMART uses the students' summary to generate a multi‐dimensional 3S (surface, structure, semantic) evaluation of the students' mental model. This model is then used to drive feedback to help students revise their summary. The current study is an initial investigation examining whether writing and revising in SMART improves students' summary quality. Students (n = 38) in a graduate‐level online course used SMART for seven reading assignments. The 38 students submitted a total of 357 summaries in response to the seven readings. In 47 cases, students produced both an initial draft and a modified revision. These 47 cases were selected for analysis. In the quantitative phase, MANOVA results indicated that students' summaries improved along the 3S dimensions from initial draft to revision. In the qualitative phase, inspection of exemplar cases revealed how students' mental models changed towards more robust and cohesive knowledge structure for texts.

Assessing students’ writing skills in virtual classroom: strategies and challenges in EFL context

“…The students' answers could be assessed by the teachers in order to collect information about students' writing skills. Summary writing is also an effective method to assess students writing (Chew et al, 2020;Kim & McCarthy, 2021). It is able to develop students' background knowledge as well as to improve their writing outcomes.…”

Formative assessment strategies for students' conceptions—The potential of learning analytics

“…Visual dashboards could provide visual summaries of student information to support the teacher in choosing successful interventions. Kim et al (2019) and Kim and McCarthy (2021) automatically extracted and visualised graph structures from students' written text to facilitate students' own reflections. They have colour‐coded the connections between concepts to indicate whether connections are (incorrectly) present or missing compared to a reference model.…”

“…Therefore, they derive linguistic properties at four different levels: descriptive, lexical, syntax and cohesion and automatically predict the students' vocabulary score using several ML algorithms. In addition, there are also approaches that attempt to externalise students' implicit knowledge by using concept maps (Giabbanelli & Tawfik, 2020; Kim et al, 2019; Kim & McCarthy, 2021; Wu et al, 2012). Wu et al (2012) compare concept maps of students with those of teachers using an existing key word list and provide hints on what students need to change.…”

“…Therefore, Giabbanelli and Tawfik (2020) use a thesaurus database in order not to depend on an exact match between student and expert concept maps. Other works even analysed the effect of automatically generated knowledge structures (KS) from text as a reflection method to improve summary writings (Kim et al, 2019; Kim & McCarthy, 2021). They compared interventions using these KS with video instructions and multiple choice questions and found, that the reflection with KS improved the similarity to expert KS the most (Kim et al, 2019; Kim & McCarthy, 2021).…”

The Role of Executive Function in an Integrated Writing Task

“…In recent years, integrated writing tasks (IW, also named writing from sources, Cumming et al, 2016) have gained popularity in language assessments, such as the Internet-based Test of English as a Foreign Language (TOEFL iBT), that aim to reflect students' written literacy in a more authentic way (Cumming et al, 2005). There are various formats under the umbrella of IW tasks in previous studies (Xie, 2023), such as summary writing based on one or more texts (Kim & McCarthy, 2021;Yang, 2014), integrated reading-to-write argumentative, informative or narrative writing tasks (Cheong et al, 2022;Leijten et al, 2019;Plakans & Gebril, 2013;Van Steendam et al, 2022;Ye & Ren, 2019), and listening-reading-writing tasks involving listening to audio before composing essays (Plakans, 2008;Zhu et al, 2016). This study was conducted in a listening-reading-writing context that required students to compose an argumentative essay for a specific purpose based on one audio and six texts.…”