Analyzing data from a pretest-posttest control group design

Abstract

Purpose -Among the gold standards in human resource development (HRD) research are studies that test theoretically developed hypotheses and use experimental designs. A somewhat typical experimental design would involve collecting pretest and posttest data on individuals assigned to a control or experimental group. Data from such a design that considered if training made a difference in knowledge, skills or attitudes, for example, could help advance practice. Using simulated datasets, situated in the example of a scenario-planning intervention, this paper aims to show that choosing a data analysis path that does not consider the associated assumptions can misrepresent findings and resulting conclusions. A review of HRD articles in a select set of journals indicated that some researchers reporting on pretest-posttest designs with two groups were not reporting associated statistical assumptions and reported results from repeated-measures analysis of variance that are considered of minimal utility. Design/methodology/approach -Using heuristic datasets, situated in the example of a scenario-planning intervention, this paper will show that choosing a data analysis path that does not consider the associated assumptions can misrepresent findings and resulting conclusions. Journals in the HRD field that conducted pretest-posttest control group designs were coded. Findings -The authors' illustrations provide evidence for the importance of testing assumptions and the need for researchers to consider alternate analyses when assumptions fail, particularly the homogeneity of regression slopes assumption. Originality/value -This paper provides guidance to researchers faced with analyzing data from a pretest-posttest control group experimental design, so that they may select the most parsimonious solution that honors the ecological validity of the data.

The Impact of Flipped Learning on Cooperative and Competitive Mindsets

“…Controlling the pre-test score as the covariate is known to increase statistical power by reducing the unexplained variance in the post-test score [76]. The mixed ANOVA (analysis of variance) were performed to confirm the basic two assumptions of the ANCOVA-the homogeneity of regression slopes and the homogeneity of variances of residual post-test scores.…”

“…The purpose of these repeated measurements of a pre-test-post-test control group design is to compare whether the flipped learning instruction had an effect on the attitudes of the students before and after the experimental intervention [72,76].…”

The impact of metacognitive practice and teacher feedback on academic achievement in mathematics

“…Normality of the data was tested using the Shapiro–Wilk test of normality and results confirmed a non‐normal distribution in the measures (Field, ). Additionally, assumption of homogeneity of variance between the pretest and posttest was met (Zientek, Nimon & Brown, ). The main threats to normality were the distributions of the skewness and kurtosis statistics in posttest and retention test scores.…”

Attitudinal Changes Toward Body-Worn Cameras: Perceptions of Cameras, Organizational Justice, and Procedural Justice Among Volunteer and Mandated Officers

“…To address this concern, we report effect size differences in Hedge's g to account for our small sample size and to present the magnitude of the effects we identify (Lakens, 2013). The reported effect sizes can also be considered as indicators of the practical significance of our findings and may prove useful for planning future studies (Zientek et al, 2016).…”