Undergraduate research experiences (UREs) in STEM fields expose students to scientific research and are thought to increase student retention in STEM. We developed a pre/post survey and administered it to participants of the Harvard Forest Summer Research Program in Ecology (HF-SRPE) to evaluate effectiveness of these programmatic goals. Between 2005 and 2015, the survey was sent to all 263 HF-SRPE participants; 79% completed it. Results, controlled for prior experiences, revealed significant improvements across all learning goals. Prior laboratory research experience and perception of being a respected member of a research team were positively associated with gains in research skills and abilities to do and present research. Although the pre/post surveys did not indicate changes in students’ goals of pursuing STEM careers (or, more narrowly, ecological ones), the positive learning gains suggest that students with prior interests in STEM fields take advantage of UREs to solidify further their aspirations in STEM.

For more than 30 years, the US National Science Foundation's Research Experiences for Undergraduates (REU) program has supported thousands of undergraduate researchers annually and provides many students with their first research experiences in field ecology or evolution. REUs embed students in scientific communities where they apprentice with experienced researchers, build networks with their peers, and help students understand research cultures and how to work within them. REUs are thought to provide formative experiences for developing researchers that differ from experiences in a college classrooms, laboratories, or field trips. REU assessments have improved through time but they are largely ungrounded in educational theory. Thus, evaluation of long‐term impacts of REUs remains limited and best practices for using REUs to enhance student learning are repeatedly re‐invented. We describe how one sociocultural learning framework, cultural–historical activity theory (CHAT), could be used to guide data collection to characterize the effects of REU programs on participant's learning in an educationally meaningful context. CHAT embodies a systems approach to assessment that accounts for social and cultural factors that influence learning. We illustrate how CHAT has guided assessment of the Harvard Forest Summer Research Program in Ecology (HF‐SRPE), one of the longest‐running REU sites in the United States. Characterizing HF‐SRPE using CHAT helped formalize thoughts and language for the program evaluation, reflect on potential barriers to success, identify assessment priorities, and revealed important oversights in data collection.