Sources of Validity Evidence for Educational and Psychological Tests

Cizek, Gregory J.; Rosenberg, Sharyn L.; Koons, Heather H.

doi:10.1177/0013164407310130

Cited by 130 publications

(98 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…If a supposedly unidimensional scale turns out to have a two-dimensional structure in a given study, then this affects the interpretation of the scale's internal structure. Note that factor-analytic evidence can be considered to be evidence of internal structure (Cizek, Rosenberg, & Koons, 2008). Therefore, we recommend that factor-analytic evidence should be presented first when assessing the internal structure of a scale.…”

Section: Factor-analytic Evidencementioning

confidence: 99%

Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all

Crutzen

Peters

2015

Health Psychology Review

196

152

View full text Add to dashboard Cite

Section: Factor-analytic Evidencementioning

confidence: 99%

Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all

Crutzen

Peters

2015

Health Psychology Review

196

152

View full text Add to dashboard Cite

“…In this paper, external aspect of validity evidence is focused upon, which deals with "convergent and discriminant evidence" (Messick, 1995, p. 745). More recently, Cizek et al (2008) examined various aspects of validity from previously published indicators. They discussed validity including the traditional division of construct validity evidence (convergent and discriminant evidence), criterionrelated evidence, content evidence, evidence based on response process, evidence based on consequences, face validity evidence and evidence based on internal structure, supporting the need for various forms of evidence.…”

Section: Introductionmentioning

confidence: 99%

Convergent and Discriminant Validity with Formative Measurement: A Mediator Perspective

Wang

French

Clay

2015

J. Mod. App. Stat. Meth.

View full text Add to dashboard Cite

show abstract

“…First, reliability and validity are difficult, complicated concepts understood by few of their users and misunderstood by many [13]- [15]. Second, very often only reliability is reported in the literature.…”

Section: Critiquementioning

confidence: 99%

“…We believe that three sources of validity evidence for the interpretation of student performance on exams can be examined relatively easily: face validity evidence, evidence based on content validity, and evidence based on internal structure [8], [15]. This is one more than the modal number of sources commonly reported in research articles aimed at establishing the sound psychometric properties of achievement, psychological, and counseling tests [15].…”

Section: ) Validity Criterionmentioning

confidence: 99%

See 1 more Smart Citation

Can Higher Education Exams Be Shortened? A Proposed Methodology

Lee¹,

Bygrave²,

Mahar³

et al. 2014

IJIET

View full text Add to dashboard Cite

Abstract-Any lecturer would agree that marking exams is the bane of her existence. A time-consuming and tiring process, it often requires complex, subjective judgments. Higher education exams typically take 3.0 hours. Do they really need to last so long? Can we justifiably reduce the number of questions on them? Shortening an exam by one hour, if justified, should result in a one-third reduction in lecturer time and effort spent marking. Surprisingly little empirical research has addressed these problems. Classical methods may be partly to blame for this dearth of studies. We propose an alternative methodology based on three key components including two recent developments in experimental design and statistics --synthetic experimental designs and equivalence hypothesis testing. The third component consists in comparing, on six psychometric criteria, student performance in a class on the standard 3.0-hr final exam with that on shortened exams with proportionately fewer questions. Two are the frequently misunderstood standard psychometric criteria -reliability and validity. We argue that adding four common-sense criteria -justifiability of test use, number of exam questions, equivalence in mean student performance, and correspondence (between shortened and full-length exam scores) -confer significant additional benefits. Our approach provides a simple methodology that lecturers can, with minimal time and effort, use to examine the effect of shortening exams for their own classes.Index Terms-Exam length, psychometric criteria, synthetic experimental designs, test length. I. INTRODUCTIONMarking exams is the bane of any lecturer's existence. It is characteristically a tedious, time-consuming process. The intricate subjective judgments that are required can exhaust even the most dedicated of lecturers. Yet despite the almost universal loathing of lecturers for this activity, surprisingly little research has been published on ways to reduce the time and effort required to mark conventional written examinations. The purpose of the present study is to redress this long-standing neglect. In this paper, we are concerned with mixed-format exams used most commonly in academe and consisting of a mixture of different types of questions including problem solving (requiring detailed solutions), Manuscript received November 6, 2013; revised January 12, 2014 essay, short answer, and multiple choice questions.Written final examinations three hours or more in length are common in many universities and colleges around the world. Why then, if marking is so universally loathed, are exams so long (both in duration and number of questions posed)? Custom or tradition seems the primary reason [1]. Even more surprisingly, long exams have been retained despite dramatic increases in class sizes over the past couple of decades at most institutions of higher learning. Many lecturers complain of class sizes more than doubling in just the last 15-20 years [2]- [4]. In effect, marking time and effort for many lecturers has, at a minimum, effective...

show abstract

Sources of Validity Evidence for Educational and Psychological Tests

Cited by 130 publications

References 9 publications

Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all

Scale quality: alpha is an inadequate estimate and factor-analytic evidence is needed first of all

Convergent and Discriminant Validity with Formative Measurement: A Mediator Perspective

Can Higher Education Exams Be Shortened? A Proposed Methodology

Contact Info

Product

Resources

About