This article describes the development of an evaluation and impact framework to assess the effectiveness of educational innovations. It can be utilized within a single program, as well as at institutional and national levels. While it is contextualized in a Chilean university, it is argued that it is widely applicable as it is informed by international best practice. The rationale that informed the development of the evaluation framework is described and is illustrated using two programs: Faculty Learning Communities; and Student Learning Assistants. These demonstrate how the framework can be customized utilizing indicators and outcomes relevant to specific programs and stakeholders.
The primary purpose of this inquiry is to analyze the impacts of a teaching and learning strategy designed and implemented by a Chilean Faculty Learning Community (FLC) intended to develop the writing competence of student-teachers of an English Teaching Program. The FLC-led strategy was implemented through an eight-step cycle based on the process-genre approach and supported by educational videos. FLC members guided this cycle during writing sessions at the four levels of the English Linguistic Competence course at Universidad Católica de Temuco. The FLC implemented this experience to address the challenge of serving diverse students’ learning needs and meet the requirements of the national English proficiency standards required by the Chilean Ministry of Education. The FLC examined this experience focusing on students’ writing tasks results and their perceptions of the use of videos in the process, oriented by an impact and evaluation framework of teaching innovations and an action research design. The ages of trainee English teachers who participated in this innovation range between 18 and 22 years old. Students’ writing tests results were analyzed and compared to the suggested CEFR outcomes per level. Moreover, students shared their perceptions towards the use of videos through focus groups. Results show that most students improved their writing performance, especially in content and organization. Furthermore, students perceived that videos helped them contextualize their writing process and contribute as a support resource embedded in classroom activities. Overall, this experience helped the FLC members identify changes resulting from the innovations and areas of improvement.
In this paper, the authors review extant natural language processing models in the context of undergraduate mechanical engineering education. These models have advanced to a stage where it has become increasingly more difficult to discern computer vs. human-produced material, and as a result, have understandably raised questions about their impact on academic integrity. As part of our review, we perform two sets of tests with OpenAI's natural language processing model (1) using GPT-3 to generate text for a mechanical engineering laboratory report and (2) using Codex to generate code for an automation and control systems laboratory. Our results show that natural language processing is a potentially powerful assistive technology for engineering students. However, it is a technology that must be used with care, given its potential to enable cheating and plagiarism behaviours given how the technology challenges traditional assessment practices and traditional notions of authorship.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.