The Benefits of Longitudinal Data and Multilevel Modeling to Measure Change in Adventure Education Research

Abstract

Background: A common critique of adventure education research methodology is the overreliance on pre-/post-study designs to measure change. Purpose: This paper compares and contrasts two methods of data analysis on the same adventure education data set to show how these distinct approaches provide starkly different results and interpretation. Methodology/Approach: Using secondary data analysis, we employ a longitudinal data set of the social skill development of urban middle school students who participated in an adventure education program over the course of three academic years. First, change was assessed using a pre-/post-design by a traditional analysis of variance (ANOVA). Next, change was assessed with a six-wave, longitudinal design using multilevel modeling. Findings/Conclusions: Results show that the multilevel modeling approach revealed nonlinear change in the social skill development of middle school students, resulting in more accurate, nuanced estimation of change in social skill development than the ANOVA approach. Implications: The use of longitudinal data and multilevel modeling can be a useful methodological approach and statistical tool for adventure education researchers to not only address the criticisms of quantitative research in adventure education, but more importantly, provide a more thorough understanding of the impact of adventure education on human development.

Untitled

“…), poor time of measures (immediately before and after the program), and small or poorly composed sample. Similar limitations in the field have been mentioned in other literature (Neill, 2003;Shirilla, Solid & Graham, 2021). However, a recent methodological paper by Shirilla, Solid and Graham (2021) noticed that current research is moving in the right direction and avoiding these limitations.…”

“…Similar limitations in the field have been mentioned in other literature (Neill, 2003;Shirilla, Solid & Graham, 2021). However, a recent methodological paper by Shirilla, Solid and Graham (2021) noticed that current research is moving in the right direction and avoiding these limitations. The authors also suggest acquiring longitudinal data, since measuring immediately before and after a program or intervention prevents us from knowing if we are measuring long-term change or just an emotional response after an exciting day.…”

A Systematic Review on Education Outside the Classroom: Lessons for Science EOC Practices

“…The lack of longitudinal data limits the conclusions one can draw regarding the impact of any EOC experience. Shirilla et al [60] argue that longitudinal data and multilevel modelling facilitate a greater understanding of pupil skill development and is necessary to progress our understanding of EOC learning outcomes [60]. It is important that long-term effects are not assumed; rigorous evaluations through long-term follow-up benefit education research [61] and should be applied to the EOC field.…”

Enhancing detection of labor violations in the agricultural sector: A multilevel generalized linear regression model of H-2A violation counts

“…This method has been employed across various disciplines to tackle diverse research questions, effectively demonstrating its worth in discerning both within-group and betweengroup effects and interactions [57]. Multilevel modeling enables researchers to accommodate intricate data structures and has found applications in various social science domains, such as education [58,59], health [60,61], environmental studies [62,63], and violence-related research [64,65].…”