Lessons and recommendations from three decades as an NSF REU site: A call for systems‐based assessment

Abstract

For more than 30 years, the US National Science Foundation's Research Experiences for Undergraduates (REU) program has supported thousands of undergraduate researchers annually and provides many students with their first research experiences in field ecology or evolution. REUs embed students in scientific communities where they apprentice with experienced researchers, build networks with their peers, and help students understand research cultures and how to work within them. REUs are thought to provide formative experiences for developing researchers that differ from experiences in a college classrooms, laboratories, or field trips. REU assessments have improved through time but they are largely ungrounded in educational theory. Thus, evaluation of long‐term impacts of REUs remains limited and best practices for using REUs to enhance student learning are repeatedly re‐invented. We describe how one sociocultural learning framework, cultural–historical activity theory (CHAT), could be used to guide data collection to characterize the effects of REU programs on participant's learning in an educationally meaningful context. CHAT embodies a systems approach to assessment that accounts for social and cultural factors that influence learning. We illustrate how CHAT has guided assessment of the Harvard Forest Summer Research Program in Ecology (HF‐SRPE), one of the longest‐running REU sites in the United States. Characterizing HF‐SRPE using CHAT helped formalize thoughts and language for the program evaluation, reflect on potential barriers to success, identify assessment priorities, and revealed important oversights in data collection.

A Contemporary Perspective on Undergraduate Research Mentoring in University Fisheries and Aquatic Sciences Programs

“…Departmental focus on undergraduate research; faculty buy-in; early student engagement Russell et al 2007;Graham et al 2013;McDevitt et al 2016;Hanauer et al 2017 Student selection Process for identification and selection Auchincloss et al 2014;McDevitt et al 2020 Setting expectations Learning contract/syllabus with measurable outcomes and clear behavioral expectations Interaction/community…”

“…Observers of this phenomenon have recently raised concerns about a lack of qualitative or quantitative assessment to help guide the evolution of UREs moving forward (Lopatto 2004;Behar-Horenstein et al 2010;Linn et al 2015;Basken 2017;Wilson et al 2018). Such haphazard and inconsistent evaluation of undergraduate research programs has even been a criticism of one of the longest running and most "successful" URE programs, the National Science Foundation's Research Experiences for Undergraduates program (McDevitt et al 2020). For the majority of the program's history, assessment was not required, and if assessment was conducted, each research site developed and implemented its own assessment instrument(s) (McDevitt et al 2016).…”

“…For the majority of the program's history, assessment was not required, and if assessment was conducted, each research site developed and implemented its own assessment instrument(s) (McDevitt et al 2016). Only recently have there been calls for standardized assessments across all participating institutions (see McDevitt et al 2020). Deriving some common elements that lead to desired outcomes for UREs, which we refer to here as "success criteria," could streamline URE implementation for others by helping them to avoid ineffective or inefficient practices.…”

Fisheries Volume 45 Number 11 November 2020

“…Departmental focus on undergraduate research; faculty buy-in; early student engagement Russell et al 2007 ;Graham et al 2013 ;McDevitt et al 2016 ;Hanauer et al 2017 Student selection Process for identifi cation and selection Auchincloss et al 2014 ;McDevitt et al 2020 Setting expectations Learning contract/syllabus with measurable outcomes and clear behavioral expectations Lown 1993 ;Gray 2000 ;Mabrouk 2003 ;Abdel-Qader 2004 Producing Skill acquisition Students receive training directly from faculty members; students are accepted and mentored by other members of the group Benson 2002 ;Gafney 2005 ;Dolan and Johnson 2009 ;Edwards et al 2011 ;Packard et al 2014 Interaction/community…”

“…For the majority of the program's history, assessment was not required, and if assessment was conducted, each research site developed and implemented its own assessment instrument(s) (McDevitt et al 2016 ). Only recently have there been calls for standardized assessments across all participating institutions (see McDevitt et al 2020 ). Deriving some common elements that lead to desired outcomes for UREs, which we refer to here as "success criteria," could streamline URE implementation for others by helping them to avoid ineffective or ineffi cient practices.…”