Formative vs. Summative ABET Assessment: A Comprehensive Graphic Representation for a New BME Program

Abstract

This paper presents an innovative methodology for the assessment of a new Biomedical Engineering (BME) program. Biomedical engineering programs are quite new in the engineering educational system. To date, in the state of Pennsylvania there are only six programs that have been accredited by ABET out of the 91 programs accredited nationwide 1 . While the guidelines of ABET are quite general and applicable to a variety of programs with different focuses, the scarcity of published data on BME specific programs poses a significant challenge on the preparation and assessment of program specific requirements. Another rather significant challenge is the limited number of students graduating from a brand new program that does not give sufficient statistical power to confirm the reliability of the assessment process.To obviate these limitations, an analysis for the assessment process has been designed that can help understand if any improvement occurs from the freshman and sophomore year (formative assessment), where concepts are introduced, to junior and senior years (summative assessment), where the concepts are reinforced/mastered and assessed again. By having a formative and summative assessment it is possible to evaluate if improvement occurs within the cohort, allowing for the redaction of a continuous improvement plan.The goal of the program has been set to have all students at a satisfactory level at the time of graduation. Having a rubric calibrated at "senior level" (i.e. master level), we found that most of the outcomes at the formative level reach a marginal outcome. This was expected as the idea is to observe if there exist learning trends between formative and summative levels where concepts are introduced, internalized and reinforced. This article presents a model that has successfully assessed student learning outcomes at one institution and that can be adapted by other programs as they prepare for accreditation.

ABET Assessment Using Blackboard Outcomes

“…The artifacts aligned with each outcome were designed specifically to satisfy each dimension of the rubric. Once aligned, these artifacts were evaluated in Blackboard Outcomes using an established four‐point rubric for each outcome and dimension (Piovesan and Vernaza 2015; Schmitz, Vernaza, and Piovesan 2016). The specific outcomes assessed, excerpted from a larger list of outcomes, were: B: An ability to design and conduct experiments as well as to analyze and interpret data D: An ability to function on multidisciplinary teams K: An ability to use techniques, skills, and modern engineering tools necessary for engineering practice L: An ability to apply in‐depth knowledge of biology M: An ability to apply knowledge of statistics …”

“…The raw data was exported from Blackboard Outcomes and processed using custom Matlab code (Matlab R2016a, MathWorks, Natick, MA) to create a radial plot of the scores (Piovesan and Vernaza 2015). Any scores lower than 3 were addressed through changes in the course.…”

Introducing a New Graduate Degree in Technology Management: Program Overview and Assessment Plan

“…Formative and summative assessment approaches are also extensively used for program assessment for ABET accreditation. A study on formative versus summative assessment for a Biomedical Engineering (BME) program concluded that most of the ABET outcomes were achieved at marginal levels by the formative assessment [4]. A good assessment process at the course level includes having Course Learning Objectives and tie them to Program Learning Objectives.…”