Measuring intangible outcomes can be problematic: The challenge of assessing learning during international short-term study experiences

Abstract

Short-term international study experiences, or study tours, aim to increase students’ global awareness, educate and empower them to be productive global citizens, and contribute to their future employability. Learning outcomes from study tours often include intangible personal characteristics or soft skills, as opposed to specific disciplinary skills and knowledge, and yet, these are not easily identified. Using an iterative Delphi process, this study aimed to understand the pedagogical experience of tour leaders (experts) and assist in future development of effective assessment. Findings include four key areas of agreement among the experts: (1) both discipline-based knowledge and the acquisition of intangible personal characteristics are important learning outcomes and are considered to be almost equally important, (2) assessment almost always occurs on tour, (3) formative assessment for learning is predominantly used (rather than learning for summative assessment to test the learning), and (4) facilitated reflection is an important way to encourage learning during a study tour. Yet, the findings also reveal uncertainty about assessing outcomes that include intangible personal characteristics and soft skills. The article concludes with pedagogical implications and recommendations for future research.

Connecting experiences to employability through a meaning-making approach to learning

“…It is important to acknowledge the problematic nature of measuring life-experience learning (Behle, 2020) and the often-intangible outcomes of experiences like international study (Cushing et al, 2019). Similarly, the challenges that students often face when undertaking reflective thinking and writing (Newcomb, Burton & Edwards, 2018) and the difficulties that academic staff have with supporting students in their endeavours and assessing reflective writing (Bourner, 2003) must also be acknowledged.…”

Factors Affecting Chinese Short‐Term International Students’ Cross‐Cultural Adaptation in Psychology, Learning, and Life

“…Besides, short-term international education, which is thought to be the key factor in promoting long-term global education, can also help students figure out the complex global issues such as aging society, gender inequality, and environmental protection. [6][7][8][9]. Additionally, based on the current short-term overseas educational development and issues, it is suggested that both sides of the universities, in collaboration, should offer students adequate support.…”

Engagement in Continuing Subject Knowledge Development: A Year After Short-Term International Courses

“…Many students explicitly commented on their learning uptake in their first interviews, but some disappointingly noted in their second interviews that they had forgotten it. SIC leaders have often emphasized subject knowledge as an outcome to the same extent as students’ personal capability growth (Cushing et al, 2017). However, the results suggested that participants’ longitudinal learning may not automatically occur regarding subject knowledge.…”

“…These studies have often probed the impact of SICs immediately after their participation. However, a lacuna exists in the literature concerning the significance of SIC outcomes in relation to students’ post-SIC engagement in their university studies (Cushing et al, 2017; Nguyen et al, 2018; Ruth et al, 2019).…”

“…Among varied formats of international education practices, faculty-directed short-term international courses (SICs) where students study in international settings for a few days or weeks have become popular (Kurt et al, 2013; Slotkin et al, 2012). Research has suggested that students develop intercultural communicative skills (Assaf et al, 2019; Sakurai, 2019), global perspectives (Cushing et al, 2017; Kurt et al, 2013), social connectedness, willingness for self-development (Weaver & Tucker, 2010), and knowledge of the visit sites (Kurt et al, 2013). However, few studies have explored how students engage in university studies drawing on their SIC outcomes.…”