Utilizing OpenAI's GPT‐4 for written feedback

Abstract

In this media review, we leverage Hubbard's (2021) framework for computer-assisted language learning (CALL) to examine the use of generative artificial intelligence (AI), specifically OpenAI's GPT-4 (fourth-generation generative pre-trained transformer), as a tool for teachers to provide their students with feedback on writing. This tripartite framework consists of an operational description of the technology, and how it fits learners' and teachers' needs and abilities. This review is therefore divided into three main components: a description of generative AI and how GPT-4 could be used in an English as a second or foreign language (ESL/EFL) teaching context, an evaluation of how well this technology fits the teacher's needs in providing feedback, and an evaluation of how well it provides useful feedback to students in order to meet their needs. | OPERATIONAL DESCRIPTIONGPT-4 is a large language model (LLM) that users can interface with through the chatbot ChatGPT (https://chat.openai.com/). GPT-4 is trained on massive amounts of text data and has been fine-tuned using reinforcement learning from human feedback (RLHF) (https://openai.com/resea rch/gpt-4). It can be accessed on mobile devices, PC, or Mac through a browser, as well as through an app for Android and iOS users. While there are LLMs that are free to use (GPT-3.5, PaLM 2), we prefer to use GPT-4 for generating feedback because it currently has superior reasoning capabilities and we have found its feedback to be more accurate, concise, and detailed. Use of GPT-4 does require a "Plus" membership (20 USD per month). Users can interact with GPT-4 by submitting a prompt (a question or request) to ChatGPT, to which the chatbot will respond. Because of high demand, prompts to GPT-4 are currently limited to 50 every 3 hours. To get the most from LLMs, effective prompt engineering is essential. Ingley and Pack (2023) offer a useful framework: assign a role or identity, define an objective, specify the context and constraints, and refine the output through continued conversation. Carefully crafted prompts will help the AI to generate more suitable and useful output. Prompt engineering is a process and a skill, and we underscore the need to refine prompts and output through continued conversation in order to increase the accuracy and appropriacy of AI-generated feedback.

ChatGPT and EFL/ESL Writing: A Systematic Review of Advantages and Challenges

“…Additionally, the findings show that the feedback and content generated by ChatGPT vary in quality, which may hinder learners' ability to rely on such tools for consistent improvement. These findings are in line with Carlson et al (2023), suggesting that feedback generated by ChatGPT is overly long or difficult for students, resulting in reduced motivation and engagement. Therefore, Carlson et al (2023) highlighted the importance of instructors not relying exclusively on ChatGPT but utilizing it as a supplementary tool.…”

“…These findings are in line with Carlson et al (2023), suggesting that feedback generated by ChatGPT is overly long or difficult for students, resulting in reduced motivation and engagement. Therefore, Carlson et al (2023) highlighted the importance of instructors not relying exclusively on ChatGPT but utilizing it as a supplementary tool. They also recommended providing students with opportunities to schedule meetings to address challenges and receive further support as necessary.…”

“…Researchers have shown that EFL/ESL learners may rely too heavily on ChatGPT, which may impair their creativity and critical thinking skills (Harunasari, 2023;Irzawati et al, 2024). Furthermore, Carlson et al (2023) observed that ChatGPT-generated feedback may be overly verbose or complex for students, reducing motivation and engagement. As a result, teachers must provide opportunities for students to meet in person to discuss concerns and obtain further assistance.…”

“…According to Eke (2023), the academic community should transparently clarify the ethical use of ChatGPT rather than outright prohibiting it, as well as update academic policies to address new challenges posed by AI in education, advocating for a balanced approach to leveraging ChatGPT's benefits while upholding academic standards. Carlson et al (2023) addressed concerns about privacy in the generated feedback, emphasizing the need for teachers to be aware of its potential biases. To this end, the current systematic review aims to explore ChatGPT's advantages and challenges in EFL/ESL writing.…”

Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process

“…Concerns have also been voiced in how AI may be used by educators (Pack & Maloney, 2023a, b;Carlson et al, 2023;Lo, 2023). Educators have used GenAI to create course material and assessment tasks, adapt materials to be more suitable for specific students, and generate lecture notes (Bonner et al, 2023;Lo, 2023).…”