Assessment of Student Learning Through Reflection on Doing in Engineering Design

Abstract

How can instructors leverage assessment instruments in design, build, and test courses to simultaneously improve student outcomes and assess student learning to improve courses? A Take-away is one type of assessment method. It is unstructured text written by a student in AME4163: Principles of Engineering Design, the University of Oklahoma, Norman, US to record what they understand by reflecting on authentic, immersive experiences throughout the semester. The immersive experiences include lectures, assignments, reviews, building, testing, and a post-analysis for the design of an electro-mechanical system to address a given customer need. In the context of a Take-away, a student then writes a Learning Statement. The Learning Statement is a single sentence written as a triple, i.e., Experience|Learning|Value. Over the past three years at the University of Oklahoma (OU), we collected about 18,000 Take-aways and 18,000 Learning Statements from almost 400 students. In our earlier papers, we primarily concentrate on analyzing students’ Learning Statements by a text mining framework. In this paper, we focus on analyzing students’ Take-aways data using a Latent Dirichlet Allocation (LDA) algorithm, and then relate the Take-away data to the instructor’s expectations using text similarity. By connecting and comparing what students learned (embodied in Take-aways) and what instructors expected the students to learn (embodied in the course booklet), we provide evidence-based guidance to instructors on improving the delivery of AME4163: Principles of Engineering Design. The proposed method can be generalized to be used for the assessment of ABET Student Outcomes 2 and 7.

Quantification of students’ active learning in design, build, and test engineering modules

“…Peng and co-authors (2020) proposed a text mining framework to facilitate the analysis of a vast number of LS obtained by engineering students and text quantification method, text similarity, to facilitate instructors gaining new insights from students' LS (Peng et al, 2020). Sun et al (2021) propose a Latent Dirichlet Allocation algorithm to analyse students' 'Takeaways (TA)' and to relate the takeaways data to instructor's expectations using text similarity.…”