Measuring Program Outcomes: Using Retrospective Pretest Methodology

Pratt, Clara C.; McGuigan, William M.; Katzev, Aphra R.

doi:10.1177/109821400002100305

Cited by 436 publications

(313 citation statements)

References 16 publications

Supporting

Mentioning

295

Contrasting

Unclassified

Order By: Relevance

“…What is more, nonequivalent dependent variables are amenable to nearly all types of research designs including, but not limited to, withinsubjects designs, between-subjects designs, single-subject designs, removed treatment designs, and interrupted time-series designs. Additionally, Nimon, Zigarmi, andAllen (2011) andPratt, McGuigan, andKatzev (2000) Source: McKillip and Baldwin, 1990, p. 341. provided some empirical evidence supporting the use of retrospective pretest, which also could usefully be combined with (retrospective) nonequivalent dependent variables in many evaluation contexts given that a large majority of evaluations are retrospective rather than prospective.…”

Section: Resultsmentioning

confidence: 97%

Using nonequivalent dependent variables to reduce internal validity threats in quasi‐experiments: Rationale, history, and examples from practice

Coryn

Hobson

2011

New Drctns Evaluation

View full text Add to dashboard Cite

Threats to the validity of inferences and conclusions regarding the effects of applied interventions have been a major dilemma for social scientists and evaluators for several decades. One mechanism for reducing threats to internal validity and improving warrants for cause-and-effect conclusions in nonrandomized investigations and evaluations is the inclusion of nonequivalent dependent variables as an element of structural design.In this chapter, the rationale for, history of, and examples from practice for using nonequivalent dependent variables to reduce internal validity threats, as well as some warrants supporting their increased use, are described.

show abstract

Section: Resultsmentioning

confidence: 97%

Using nonequivalent dependent variables to reduce internal validity threats in quasi‐experiments: Rationale, history, and examples from practice

Coryn

Hobson

2011

New Drctns Evaluation

View full text Add to dashboard Cite

show abstract

“…The retrospective pretest-posttest experimental model was used in the study. After completing the education program in this model, the conditions of the participants at the end of the program and before their participation in the program are interrogated at the same time (Sprangers, & Hoogstraten, 1989;Pratt, McGuigan & Katzev, 2000). In this model, they are requested to first evaluate their conditions at the end of the education program and then their conditions in the past (Rohs, 1999).…”

Section: Discussionmentioning

confidence: 99%

The Effectiveness of Leadership Skills Development Program for University Students / Üniversite Öğrencilerine Yönelik Liderlik Becerileri Geliştirme Programının Etkililiği

Cansoy

2017

TAKSAD

View full text Add to dashboard Cite

The aim of this study was to examine the effectiveness of a leadership skills education program for higher education students. In this program, education was provided to improve the desire for struggle and goal setting, communication skills, group skills, problem-solving skills, decision-making skills, responsibility awareness, trusting and trustworthiness awareness, leadership awareness and emotional awareness within the framework of leadership skills. The study group of the research consists of finalyear students with the average age of 23.8 years studying at the Faculty of Economics of Karabük University in the 2016-2017 academic year. The data of the study were collected with the "Youth leadership characteristics scale", "Emotional self-awareness scale" and "Awareness of leader and leadership scale". The retrospective pretest-posttest experimental model was used in the study. The differentiation levels of the pretest and posttest scores of the students included in the education program were examined with the Wilcoxon signed-rank test. It was observed at the end of the study that the desire for struggle and goal setting, communication skills, group skills, problem-solving skills, decision-making skills, responsibility awareness, trusting and trustworthiness awareness, leadership awareness and emotional awareness levels of the students who participated in the education program significantly increased from a moderately sufficient level to a quite sufficient level. It was indicated that the applied leadership skills education program was effective.

show abstract

“…To address these issues and others not specific to child welfare research, scholars have advocated for the use of ''retrospective pretests'' (aka, thentests) to measure baseline knowledge, skills, or attitudes. Although researchers have discussed the utility and validity of retrospective pretests for over 60 years (Deutsch and Collins 1951;Walk 1956), the debate continues as this research tactic is adopted by different disciplines for various types of evaluations (English and Horowitz 2002;Piwowar and Thiel 2014;Pratt et al 2000). Evidence for or against the use of retrospective pretests in child welfare research is particularly needed, as their application in this setting is less common in practice and underreported in the peer-reviewed literature.…”

Section: Introductionmentioning

confidence: 96%

The Use of Prospective Versus Retrospective Pretests with Child-Welfare Involved Families

et al. 2016

View full text Add to dashboard Cite

The purpose of the current study is to examine the use of both a prospective and a retrospective pretest in evaluating the impact of the Strengthening Families Program on foster care involved families affected by parental substance abuse. Debate has existed in the literature for over 60 years regarding the use of retrospective pretests in assessing self-reported program impact, with key stakeholders often arguing strongly that, due to response-shift bias, sensitization, and the nature of many human service settings, retrospective pretesting may be more appropriate. However, program evaluators must also incorporate into the evaluation design funding mandates to collect data at specific points in time. In order to mitigate potential pretest-only biases and enhance the evaluation's rigor, this study sought to address the pretest debate through the use of both a traditional, prospective pretest and a retrospective pretest. Using data provided by 411 caregivers, program effectiveness was measured in the areas of family, child and parent functioning. Statistical significance tests and effect sizes were analyzed to compare traditional prospective pretest to posttest scores and retrospective pretests to posttest scores. Findings indicate that overall, the Strengthening Families Program positively impacted family, child and parent functioning and that there were few differences between testing approaches when testing for statistical significance; however, relative differences between prospective pretests and retrospective pretests appeared more prominent in effect size computations. This research informs the longstanding debate, and suggests that program evaluators consider the advantages and disadvantages of using a retrospective pretest in design planning.

show abstract

Measuring Program Outcomes: Using Retrospective Pretest Methodology

Cited by 436 publications

References 16 publications

Using nonequivalent dependent variables to reduce internal validity threats in quasi‐experiments: Rationale, history, and examples from practice

Using nonequivalent dependent variables to reduce internal validity threats in quasi‐experiments: Rationale, history, and examples from practice

The Effectiveness of Leadership Skills Development Program for University Students / Üniversite Öğrencilerine Yönelik Liderlik Becerileri Geliştirme Programının Etkililiği

The Use of Prospective Versus Retrospective Pretests with Child-Welfare Involved Families

Contact Info

Product

Resources

About