The Reliability of Test Scores

Kelley, Truman L.

doi:10.1080/00220671.1921.10879169

Cited by 37 publications

(21 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The U coefficient (and the u coefficient) can also be obtained from the reliability coefficient of the unrestricted group and the reliability coefficient of the restricted group. Based on Kelley's () formula,

S D \sqrt{1 - R_{x x}} = s d \sqrt{1 - r_{x x}}

and, by carrying out some algebraic transformations, it is possible to obtain the following two equations:

\frac{S D}{s d} = \frac{\sqrt{1 - r_{x x}}}{\sqrt{1 - R_{x x}}} = U

and

\frac{s d}{S D} = \frac{\sqrt{1 - R_{x x}}}{\sqrt{1 - r_{x x}}} = u

…”

Section: A New Theoretical Model Of Faking Effectsmentioning

confidence: 99%

“…Both 1‐R xx and 1‐r xx terms are error variance. Kelley (), Otis (), Brogden (), Hunter et al (), and many other researchers, have suggested that it is reasonable to assume that the error variance is equal for the true scores in the unrestricted and restricted groups. They also suggested that it can also be assumed that it is uncorrelated with all selections which can produce changes in the variance of Y .…”

Section: A New Theoretical Model Of Faking Effectsmentioning

confidence: 99%

See 1 more Smart Citation

A Theoretical Model of Psychometric Effects of Faking on Assessment Procedures: Empirical findings and implications for personality at work

Salgado

2016

Int J Selection Assessment

View full text Add to dashboard Cite

This article proposes a theoretical model for explaining the psychometric effects of faking on assessment procedures (e.g., biodata, interviews, assessment center, personality inventories, and self‐reported measures). The model hypothesizes that faking is a phenomenon of homogenization of scores, consisting of a double mechanism that increases the mean, on one hand, and decreases the standard deviation of distributions of scores, on the other. Subsequently, this affects the covariance, reliability, and validity of assessment procedures negatively. The model predicts that a mean ratio (faking ratio) greater than 1 and a coefficient of homogeneity u smaller than 1 characterizes faking. Meta‐analysis was used to test several predictions of the theoretical model in the case of the personality measures. A database of 46 independent studies was created. All the studies used the NEO‐PI‐R for assessing the Big Five and their facets. The general pattern of data fully supported the model predictions. Implications for personnel assessment and, particularly, for personality assessment are discussed.

show abstract

S D \sqrt{1 - R_{x x}} = s d \sqrt{1 - r_{x x}}

and, by carrying out some algebraic transformations, it is possible to obtain the following two equations:

\frac{S D}{s d} = \frac{\sqrt{1 - r_{x x}}}{\sqrt{1 - R_{x x}}} = U

and

\frac{s d}{S D} = \frac{\sqrt{1 - R_{x x}}}{\sqrt{1 - r_{x x}}} = u

…”

Section: A New Theoretical Model Of Faking Effectsmentioning

confidence: 99%

Section: A New Theoretical Model Of Faking Effectsmentioning

confidence: 99%

A Theoretical Model of Psychometric Effects of Faking on Assessment Procedures: Empirical findings and implications for personality at work

Salgado

2016

Int J Selection Assessment

View full text Add to dashboard Cite

show abstract

“…Her statistical analyses were sophisticated for her time, drawing, for example, on the work of Henry Garrett (1926), Karl Holzinger (1923), Truman Kelley (1921), and Donald Paterson (1925), and allowed her to measure reliability both within and between tests. Split-half reliabilities were quite high, ranging from .91 to .93 for the four tests, whereas the intercorrelations among the various tests ranged from .55 to .65, with one outlier of .19 (true-false and matching).…”

Section: Graduate Study At the University Of Coloradomentioning

confidence: 99%

Inez Beverly Prosser and the education of African Americans

Benjamin

McMahon

2005

J. Hist. Behav. Sci.

View full text Add to dashboard Cite

Inez Beverly Prosser (ca. 1895-1934) was arguably the first African American woman to earn a doctorate in psychology. Her dissertation, completed in 1933, examined personality differences in black children attending either voluntarily segregated or integrated schools and concluded that black children were better served in segregated schools. This research was one of several studies in the 1920s and 1930s that was part of the debate on segregated schools as maintained in the United States under the "separate but equal" doctrine of Plessy v. Ferguson (1896). This article examines the life and career of Prosser in the context of educational barriers and opportunities for African Americans in the early part of the twentieth century and explores the arguments that pitted African Americans against one another in determining how best to educate black children, arguments that eventually led to the desegregation decision of Brown v. Board of Education (1954).

show abstract

“…Moreover, it should be taken into account that if the ratings are restricted in range Murphy and De Shon should have attenuated the reliability proportionally in order to correct for unreliability. This can be done using the formula developed independently each other by Otis (1922) and Kelley (1921) and reproduced in many books and articles (see the Otis-Kelley formula in the Appendix). The application of this formula would result in an alpha coefficient of .68 in the first case and of -.26 in the second.…”

Section: Is 52 a Dubious Interrater Reliability Value?mentioning

confidence: 99%

Corrections for criterion reliability in validity generalization: The consistency of Hermes, the utility of Midas

Salgado¹,

Moscoso²,

Anderson³

2016

Revista de Psicología del Trabajo y de las Organizaciones

View full text Add to dashboard Cite

a b s t r a c tThere is criticism in the literature about the use of interrater coefficients to correct for criterion reliability in validity generalization (VG) studies and disputing whether .52 is an accurate and non-dubious estimate of interrater reliability of overall job performance (OJP) ratings. We present a second-order meta-analysis of three independent meta-analytic studies of the interrater reliability of job performance ratings and make a number of comments and reflections on LeBreton et al.'s paper. The results of our meta-analysis indicate that the interrater reliability for a single rater is .52 (k = 66, N = 18,582, SD = .105). Our main conclusions are: (a) the value of .52 is an accurate estimate of the interrater reliability of overall job performance for a single rater; (b) it is not reasonable to conclude that past VG studies that used .52 as the criterion reliability value have a less than secure statistical foundation; (c) based on interrater reliability, test-retest reliability, and coefficient alpha, supervisor ratings are a useful and appropriate measure of job performance and can be confidently used as a criterion; (d) validity correction for criterion unreliability has been unanimously recommended by "classical" psychometricians and I/O psychologists as the proper way to estimate predictor validity, and is still recommended at present; (e) the substantive contribution of VG procedures to inform HRM practices in organizations should not be lost in these technical points of debate.

show abstract

The Reliability of Test Scores

Cited by 37 publications

References 0 publications

A Theoretical Model of Psychometric Effects of Faking on Assessment Procedures: Empirical findings and implications for personality at work

A Theoretical Model of Psychometric Effects of Faking on Assessment Procedures: Empirical findings and implications for personality at work

Inez Beverly Prosser and the education of African Americans

Corrections for criterion reliability in validity generalization: The consistency of Hermes, the utility of Midas

Contact Info

Product

Resources

About