Effects of Coaching on SAT I: Reasoning Test Scores

Powers, Donald E.; Rock, Donald A.

doi:10.1111/j.1745-3984.1999.tb00549.x

Cited by 94 publications

(91 citation statements)

References 28 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…No se trata de inferir que los entrenamientos cortos podrían incrementar las habilidades de razonamiento, sino que la preparación en destrezas y estrategias para realizar estas pruebas podría producir efectos positivos en los puntajes de las mismas (Cohen, 2006;Shulman, 2013;Burns, Siers & Christiansen, 2008;Martínez-Cardenoso, Muniz & García Cueto, 2000;Powers & Rock, 1999). Esta discusión adquiere mayor relevancia para Costa Rica, bajo un contexto de inequidad creciente en el acceso a las oportunidades educativas (Fernández & Del Valle, 2013;Montero, Rojas, Rodino & Zamora, 2012;Trejos, 2010), y de ahí que sea evidente la necesidad de documentar científicamente la posibilidad de efectos de capacitación potenciales, no directamente relacionados al constructo que se pretende medir con la prueba de admisión.…”

Section: Introductionunclassified

Efecto de una capacitación sobre los puntajes de la prueba de admisión de la Universidad de Costa Rica: una aproximación bayesiana

Rojas

Rojas-Rojas

Hernández

et al. 2015

View full text Add to dashboard Cite

Efecto de una capacitación sobre los puntajes de la prueba de admisión de la Universidad de Costa Rica: una aproximación bayesiana * Resumen. Se usó un diseño cuasi-experimental con pre y post-test para estimar el efecto de una capacitación para la prueba de admisión de la Universidad Costa Rica, un test estandarizado que mide habilidades de razonamiento en contextos verbales y matemáticos. Cuatro colegios públicos del área metropolitana central del país participaron en el estudio, asignándose dos de ellos aleatoriamente al grupo de intervención y los otros dos al grupo de control, con 61 estudiantes en el primer grupo y 80 en el segundo. La intervención consistió de 5 sesiones de capacitación de 3 horas, utilizando como guía un manual desarrollado por una experta pedagoga, con enfoque constructivista. Las medidas antes y después fueron formas reducidas de la prueba de admisión 2014. La variable dependiente fue la diferencia entre ambas mediciones. El efecto de la capacitación fue de 3.5 puntos porcentuales y significativo, y se estimó utilizando un modelo bayesiano de regresión multinivel.Palabras clave. Evaluación de impacto, modelos jerárquicos bayesianos, efectos de capacitación, prueba de admisión a la Universidad, modelos multinivel. Abstract.A quasi-experimental design with pre and post-test was used to estimate training effects for the University of Costa Rica's admission test, a standardized exam that measures reasoning abilities in mathematical and verbal contexts. Four secondary public schools from the metropolitan central area of the country participated in the study; two of them were randomly assigned to the intervention group and the other two to the control group, with 61 students in the first group and 80 in the second. The intervention consisted of five three-hour training sessions, using a written guide developed by a pedagogy expert with a constructivist approach. Before and after measures were reduced test forms of the real admission test from the year 2014. The dependent variable was the difference between the two measures. The effect of the training was estimated using a multilevel Bayesian regression model with a significant magnitude of 3.5 percentage points.

show abstract

Section: Introductionunclassified

Efecto de una capacitación sobre los puntajes de la prueba de admisión de la Universidad de Costa Rica: una aproximación bayesiana

Rojas

Rojas-Rojas

Hernández

et al. 2015

View full text Add to dashboard Cite

show abstract

“…Powers and Rock (1999) more recently found that coaching raised scores on the revised version of the Scholastic Assessment Test by less than 0.10 and 0.20 on the verbal and math portions, respectively.…”

Section: Practice and Testingmentioning

confidence: 99%

Retaking ability tests in a selection setting: Implications for practice effects, training performance, and turnover.

Hausknecht¹,

Trevor²,

Farr³

2002

Journal of Applied Psychology

View full text Add to dashboard Cite

This field study investigated the effect of retaking identical selection tests on subsequent test scores of 4,726 candidates for law enforcement positions. For both cognitive ability and oral communication ability selection tests, candidates produced significant score increases between the 1st and 2nd and the 2nd and 3rd test administrations. Furthermore, the repeat testing relationships with posthire training performance and turnover were examined in a sample of 1,515 candidates eventually selected into the organization. As predicted from persistence and continuance commitment rationales, the number of tests necessary to gain entry into the organization was positively associated with training performance and negatively associated with turnover probability. the 1st and 2nd and the 2nd and 3rd test administrations. Furthermore, the repeat testing relationships with posthire training performance and turnover were examined in a sample of 1,515 candidates eventually selected into the organization. As predicted from persistence and continuance commitment rationales, the number of tests necessary to gain entry into the organization was positively associated with training performance and negatively associated with turnover probability. Retaking Ability Tests 3The area of employment-related test preparation is undergoing considerable growth.Moreover, many employers, particularly those in the public sector, allow job applicants to retake employee selection tests. Thus, it is important to gauge, in the employment realm, the effectiveness and implications of what is commonly referred to as test coaching and practice.Although some research has addressed whether coaching and practice generally yield improvements in ability test scores (Kulik, Bangert-Drowns, & Kulik, 1984; and although more recent studies have examined test preparation courses in such contexts as college admission decisions (e.g., Briggs, 2001), no research exists on ability test practice in an employee selection context. Consequently, the important issue of whether retaking such selection tests might change test scores and subsequent hiring decisions has been, as of yet, unexplored. In addition, retaking these tests may have implications that extend beyond employee selection. In this study, we examined cognitive ability and oral communication ability selection tests to understand how scores change with repeat administrations. Furthermore, we investigated whether retaking such tests is related to posthire training performance and employee turnover.Thus, the purpose of this article is threefold. First, we examined the relationship between test practice and test scores. Drawing from research in educational psychology and other fields, we attempted to extend to employment settings the prior findings concerning practice effects on cognitive ability test scores. We also investigated a potential practice effect on oral communication ability test scores for the same sample. Second, we investigated the relationship between the number of attempts to gain entr...

show abstract

“…The data to be analyzed derive from a stratified random sample of registrants for 1995-1996 administrations of the SAT-I test, details of which are given by Powers and Rock (1999). About 6,700 high school juniors and seniors received surveys asking whether and how they had prepared for the test; the replies of some 4,200 respondents were linked to the College Board's records of their scores on the 1995 or 1996 exams, as well as scores on previous SAT-I or PSAT tests and their answers to the Student Descriptive Questionnaire (SDQ), which all SAT-I registrants are asked to complete.…”

Section: Test Scores and Test Preparation In A National Samplementioning

confidence: 99%

“…Some 12% of respondents said that they had; the comparison of these to the remaining 88% comprised the observational study reported by Powers and Rock (1999).…”

Section: Introductionmentioning

confidence: 99%

Full Matching in an Observational Study of Coaching for the SAT

Hansen

2004

Journal of the American Statistical Association

474

466

View full text Add to dashboard Cite

Among matching techniques for observational studies, full matching is in principle the best, in the sense that its alignment of comparable treated and control subjects is as good as that of any alternate method, and potentially much better. This article evaluates the practical performance of full matching for the first time, modifying it in order to minimize variance as well as bias and then using it to compare coached and uncoached takers of the SAT. In this new version, with restrictions on the ratio of treated subjects to controls within matched sets, full matching makes use of many more observations than does pair matching, but achieves far closer matches than does matching with k ≥ 2 controls. Prior to matching, the coached and uncoached groups are separated on the propensity score by 1.1 SDs. Full matching reduces this separation to 1% or 2% of an SD. In older literature comparing matching and regression, Cochran expressed doubts that any method of adjustment could substantially reduce observed bias of this magnitude.To accommodate missing data, regression-based analyses by ETS researchers rejected a subset of the available sample that differed significantly from the subsample they analyzed. Full matching on the propensity score handles the same problem simply and without rejecting observations. In addition, it eases the detection and handling of nonconstancy of treatment effects, which the regression-based analyses had obscured, and it makes fuller use of covariate information. It estimates a somewhat larger effect of coaching on the math score than did ETS's methods.

show abstract

Effects of Coaching on SAT I: Reasoning Test Scores

Cited by 94 publications

References 28 publications

Efecto de una capacitación sobre los puntajes de la prueba de admisión de la Universidad de Costa Rica: una aproximación bayesiana

Efecto de una capacitación sobre los puntajes de la prueba de admisión de la Universidad de Costa Rica: una aproximación bayesiana

Retaking ability tests in a selection setting: Implications for practice effects, training performance, and turnover.

Full Matching in an Observational Study of Coaching for the SAT

Contact Info

Product

Resources

About