Mitigating Ceiling Effects in a Longitudinal Study of Doctoral Engineering Student Stress and Persistence

Abstract

Aim/Purpose: The research reported here aims to demonstrate a method by which novel applications of qualitative data in quantitative research can resolve ceiling effect tensions for educational and psychological research. Background: Self-report surveys and scales are essential to graduate education and social science research. Ceiling effects reflect the clustering of responses at the highest response categories resulting in non-linearity, a lack of variability which inhibits and distorts statistical analyses. Ceiling effects in stress reported by students can negatively impact the accuracy and utility of the resulting data. Methodology: A longitudinal sample example from graduate engineering students’ stress, open-ended critical events, and their early departure from doctoral study considerations demonstrate the utility and improved accuracy of adjusted stress measures to include open-ended critical event responses. Descriptive statistics are used to describe the ceiling effects in stress data and adjusted stress data. The longitudinal stress ratings were used to predict departure considerations in multilevel modeling ANCOVA analyses and demonstrate improved model predictiveness. Contribution: Combining qualitative data from open-ended responses with quantitative survey responses provides an opportunity to reduce ceiling effects and improve model performance in predicting graduate student persistence. Here, we present a method for adjusting stress scale responses by incorporating coded critical events based on the Taxonomy of Life Events, the application of this method in the analysis of stress responses in a longitudinal data set, and potential applications. Findings: The resulting process more effectively represents the doctoral student experience within statistical analyses. Stress and major life events significantly impact engineering doctoral students’ departure considerations. Recommendations for Practitioners: Graduate educators should be aware of students’ life events and assist students in managing graduate school expectations while maintaining progress toward their degree. Recommendation for Researchers: Integrating coded open-ended qualitative data into statistical models can increase the accuracy and representation of the lived student experience. The new approach improves the accuracy and presentation of students’ lived experiences by incorporating qualitative data into longitudinal analyses. The improvement assists researchers in correcting data with ceiling effects for use in longitudinal analyses. Impact on Society: The method described here provides a framework to systematically include open-ended qualitative data in which ceiling effects are present. Future Research: Future research should validate the coding process in similar samples and in samples of doctoral students in different fields and master’s students.