An examination of psychometric bias due to retesting on cognitive ability tests in selection settings.

Lievens, Filip; Reeve, Charlie L.; Heggestad, Eric D.

doi:10.1037/0021-9010.92.6.1672

Cited by 87 publications

(135 citation statements)

References 45 publications

(62 reference statements)

Supporting

Mentioning

115

Contrasting

Unclassified

Order By: Relevance

“…After exploratory factor analyses and to analyze whether the SAQ has a stable and consistent factor structure, we analyzed configural and metric invariance (Lievens, Reeve, & Heggestad, 2007;Vandenberg & Lance, 2000). For configural invariance (whether the series of items reflects the same latent construct in the diverse groups), we used Tucker's congruence coefficient (r c ), following the recommendations of Jensen (1998, pp.…”

Section: Discussionmentioning

confidence: 99%

Temporal Stability and Cross-National Consistency of the Dimensional Structure of the Sexual Attraction Questionnaire (SAQ)

et al. 2009

View full text Add to dashboard Cite

The Sexual Attraction Questionnaire (SAQ) was designed to measure sexual attraction (Fernández, Quiroga, & Rodríguez, 2006), because the current questionnaires were considered inadequate. The purpose of this research was to test whether the SAQ factors remain meaningful after several years (stability) and whether the Italian version is equivalent to the Spanish one (consistency). Three groups of university students participated: two from Spain (n = 182 and 255, respectively) and one from Italy (n = 293). The Spanish groups were tested with a 7-year interval (2001)(2002)(2003)(2004)(2005)(2006)(2007)(2008). The Italian group was tested in 2008. The main hypotheses were to test, across time interval and countries: (a) factor congruence, (b) predictive power of factors (proportion of variance accounted for), and (c) scale reliability. Sexual attraction typology also was analyzed, within and between countries, to test the validity of the underlying theoretical model. The results obtained show that the SAQ factor structure remains the same, the resulting factors have high predictive power, and the SAQ scales are highly reliable. Sexual dimorphism and sexual attraction typology are highly associated, thus validating the underlying theoretical model. Keywords: sexual attraction, sexual orientation, sexual typology, factor congruence and stability, cross-national studies.El Cuestionario de Atracción Sexual (CAS) se creó para valorar la atracción sexual (Fernández, Quiroga, & Rodríguez, 2006) debido a la ausencia de instrumentos específicos. Se pretende analizar ahora si la estructura dimensional del CAS permanece estable con el paso del tiempo y si es semejante en la versión italiana. Participaron tres grupos de estudiantes universitarios: dos españoles (182 y 255) y uno italiano (293). Los datos del primer grupo español se obtuvieron en 2001 y los del segundo en 2008, junto con los del grupo italiano. Las hipótesis se centran en analizar: (a) la congruencia de las estructuras factoriales; (b) el poder predictivo de los factores (proporción de varianza explicada) y (c) la fiabilidad de las escalas del CAS, tanto entre países como entre los dos momentos temporales. Además, para poner a prueba la validez del modelo teórico subyacente, se analiza la semejanza en la tipología de la atracción sexual intra e inter-países. Los resultados muestran que la estructura factorial del CAS se mantiene, sus factores poseen un alto poder predictivo y las escalas gozan de alta fiabilidad. El dimorfismo sexual y la tipología de atracción sexual están altamente relacionados, lo que apoya la validez del modelo teórico subyacente. Palabras clave: atracción sexual, orientación sexual, tipología sexual, congruencia y estabilidad factorial, estudios interpaíses.

show abstract

Section: Discussionmentioning

confidence: 99%

Temporal Stability and Cross-National Consistency of the Dimensional Structure of the Sexual Attraction Questionnaire (SAQ)

et al. 2009

View full text Add to dashboard Cite

show abstract

“…But while scores may be rising across test administrations, the actual ability level is assumed to be relatively stable. From a psychometric point of view, several researchers have shown that retesting induces measurement bias that makes it difficult to compare initial test and retest scores (e.g., Lievens, Reeve, & Heggestad, 2007). Individual test takers are therefore expected to make assessments that are affected by a number of psychometric problems, the concepts of which are difficult to understand for nonexperts.…”

Section: Critical Aspects Of the Present Meta-analysismentioning

confidence: 99%

How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability.

Freund¹,

Kasten²

2012

Psychological Bulletin

229

163

View full text Add to dashboard Cite

Individuals' perceptions of their own level of cognitive ability are expressed through self-estimates. They play an important role in a person's self-concept because they facilitate an understanding of how one's own abilities relate to those of others. People evaluate their own and other persons' abilities all the time, but self-estimates are also used in formal settings, such as, for instance, career counseling. We examine the relationship between self-estimated and psychometrically measured cognitive ability by conducting a random-effects, multilevel meta-analysis including a total of 154 effect sizes reported in 41 published studies. Moderator variables are specified in a mixed-effects model both at the level of the individual effect size and at the study level. The overall relationship is estimated at r ϭ .33. There is significant heterogeneity at both levels (i.e., the true effect sizes vary within and between studies), and the results of the moderator analysis show that the validity of self-estimates is especially enhanced when relative scales with clearly specified comparison groups are used and when numerical ability is assessed rather than general cognitive ability. The assessment of less frequently considered dimensions of cognitive ability (e.g., reasoning speed) significantly decreases the magnitude of the relationship. From a theoretical perspective, Festinger's (1954) theory of social comparison and Lecky's (1945) theory of selfconsistency receive empirical support. For practitioners, the assessment of self-estimates appears to provide diagnostic information about a person's self-concept that goes beyond a simple "test-and-tell" approach. This information is potentially relevant for career counselors, personnel recruiters, and teachers.

show abstract

“…Hembree (1988) reported the effect of test-skills training on actual test performance to be an r equivalent of .13, and Ergene (2003) reported an r equivalent of .21. Please note, we are not suggesting that practice effects are g-loaded (see, Lievens, Reeve, & Heggestad, 2007;te Nijenhuis et al, 2007). Rather, we are simply noting that g tends to be correlated with learning outcomes in all domains, we thus assume that in the context of test-skills training, g would also be related to skill/knowledge acquisition.…”

Section: G-test Familiarity Relationship (R Gf )mentioning

confidence: 92%

Modeling the impact of test anxiety and test familiarity on the criterion-related validity of cognitive ability tests

Reeve

Heggestad

Lievens³

2009

Intelligence

Self Cite

View full text Add to dashboard Cite

The assessment of cognitive abilities, whether it is for purposes of basic research or applied decision making, is potentially susceptible to both facilitating and debilitating influences. However, relatively little research has examined the degree to which these factors might moderate the criterion-related validity of cognitive ability tests. To address this gap, we use Classical Test Theory formulas to articulate how test anxiety and test familiarity can influence observed scores, observed score variance, and most importantly, the criterion-related validity of observed scores. The resulting equations reveal that understanding the influence of test anxiety and test familiarity on criterion-related validity coefficients requires the consideration of a number of additional parameters. To elucidate the implications of the model, we present a Monte Carlo simulation. Results show that anxiety and familiarity can have a significant negative effect on the observed criterion-related validity, but also show that this effect is highly variable. In particular, the effect depends heavily upon the relation between these factors and the criterion variable. Additionally, we note that the equations we develop highlight important gaps in the literature; there are few clear empirical estimates of several of the parameters in our formulas. We call for future research to better examine these additional relations.

show abstract

An examination of psychometric bias due to retesting on cognitive ability tests in selection settings.

Cited by 87 publications

References 45 publications

Temporal Stability and Cross-National Consistency of the Dimensional Structure of the Sexual Attraction Questionnaire (SAQ)

Temporal Stability and Cross-National Consistency of the Dimensional Structure of the Sexual Attraction Questionnaire (SAQ)

How smart do you think you are? A meta-analysis on the validity of self-estimates of cognitive ability.

Modeling the impact of test anxiety and test familiarity on the criterion-related validity of cognitive ability tests

Contact Info

Product

Resources

About