A Study of Fatigue Effects From the New Sat®

Liu, Jinghua; Allspach, Jill R.; Feigenbaum, Miriam; Oh, Hyeon-Joo; Burton, Nancy

doi:10.1002/j.2333-8504.2004.tb01973.x

Cited by 11 publications

(12 citation statements)

References 9 publications

(11 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…10 Compute coefficient alpha to estimate the reliabilities of shortened exams and the confidence intervals for alpha. Minimal cost to conduct the study.…”

Section: Discussionmentioning

confidence: 99%

“…10 Our four proposed psychometric criteria are easily understood, common sense requirements compared with the frequently misunderstood classic criteria of reliability and validity. 11…”

Section: Discussionmentioning

confidence: 99%

“…Two recent papers typify the approaches frequently used to investigate these issues. To study the effect of different SAT test lengths on performance and subjective fatigue effects, [10] employed a conventional independent-groups experimental design while [11] employed a conventional dependent-groups experimental design. While both of these study designs were appropriate to examine the specific questions addressed by these authors, they are also time-consuming, complex, and difficult to conduct.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Can Higher Education Exams Be Shortened? A Proposed Methodology

Lee¹,

Bygrave²,

Mahar³

et al. 2014

IJIET

View full text Add to dashboard Cite

Abstract-Any lecturer would agree that marking exams is the bane of her existence. A time-consuming and tiring process, it often requires complex, subjective judgments. Higher education exams typically take 3.0 hours. Do they really need to last so long? Can we justifiably reduce the number of questions on them? Shortening an exam by one hour, if justified, should result in a one-third reduction in lecturer time and effort spent marking. Surprisingly little empirical research has addressed these problems. Classical methods may be partly to blame for this dearth of studies. We propose an alternative methodology based on three key components including two recent developments in experimental design and statistics --synthetic experimental designs and equivalence hypothesis testing. The third component consists in comparing, on six psychometric criteria, student performance in a class on the standard 3.0-hr final exam with that on shortened exams with proportionately fewer questions. Two are the frequently misunderstood standard psychometric criteria -reliability and validity. We argue that adding four common-sense criteria -justifiability of test use, number of exam questions, equivalence in mean student performance, and correspondence (between shortened and full-length exam scores) -confer significant additional benefits. Our approach provides a simple methodology that lecturers can, with minimal time and effort, use to examine the effect of shortening exams for their own classes.Index Terms-Exam length, psychometric criteria, synthetic experimental designs, test length. I. INTRODUCTIONMarking exams is the bane of any lecturer's existence. It is characteristically a tedious, time-consuming process. The intricate subjective judgments that are required can exhaust even the most dedicated of lecturers. Yet despite the almost universal loathing of lecturers for this activity, surprisingly little research has been published on ways to reduce the time and effort required to mark conventional written examinations. The purpose of the present study is to redress this long-standing neglect. In this paper, we are concerned with mixed-format exams used most commonly in academe and consisting of a mixture of different types of questions including problem solving (requiring detailed solutions), Manuscript received November 6, 2013; revised January 12, 2014 essay, short answer, and multiple choice questions.Written final examinations three hours or more in length are common in many universities and colleges around the world. Why then, if marking is so universally loathed, are exams so long (both in duration and number of questions posed)? Custom or tradition seems the primary reason [1]. Even more surprisingly, long exams have been retained despite dramatic increases in class sizes over the past couple of decades at most institutions of higher learning. Many lecturers complain of class sizes more than doubling in just the last 15-20 years [2]- [4]. In effect, marking time and effort for many lecturers has, at a minimum, effective...

show abstract

“…10 Compute coefficient alpha to estimate the reliabilities of shortened exams and the confidence intervals for alpha. Minimal cost to conduct the study.…”

Section: Discussionmentioning

confidence: 99%

“…10 Our four proposed psychometric criteria are easily understood, common sense requirements compared with the frequently misunderstood classic criteria of reliability and validity. 11…”

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Can Higher Education Exams Be Shortened? A Proposed Methodology

Lee¹,

Bygrave²,

Mahar³

et al. 2014

IJIET

View full text Add to dashboard Cite

show abstract

“…Many studies in the literature (e.g., Johnson, Penny, & Gordon, 2001;Liu, Allspach, Feigenbaum, Oh, & Burton, 2004) have indicated that at least 2 raters should score students' writing assessments to improve inter-rater reliability. However, even for assessments that characteristically demonstrate high levels of rater agreement, 2 raters scoring the same essay can occasionally report different, or discrepant, scores (Johnson et al, 2001).…”

mentioning

confidence: 99%

“…Some researchers such as Weigle (1994), andMcNamara andWesley (1996) agree on the fact that with careful monitoring and training of raters, scoring procedure might end in reliable results and unbiased judgments. Too, some scholars (e.g., Cumming, 1954;Bendig, 1955;Drave, 2011;Tucker, 1948;Massey, 1977;Wohlhueter, 1966;Liu et al, 2004) maintain that construct-irrelevant factors, fatigue in particular, do not significantly affect test test-takers' scores and test-givers' scoring method. Bendig (1955), as a case in point, investigated the reliability of rater scoring and its possible loss as a result of fatigue and suggested that judgment fatigue did not affect scoring reliability.…”

mentioning

confidence: 99%

Raters’ Fatigue and Their Comments During Scoring Writing Essays: A Case of Iranian Efl Learners

Mahshanian¹,

Eslami²,

Ketabi³

2017

Indonesian J. Appl. Linguist

View full text Add to dashboard Cite

Rating accuracy in writing among EFL learners is crucial in determining their English proficiency. Despite the importance of its accuracy, little is known about various factors may affect the accuracy of rating writing essays. This study examines how raters' comments on EFL writing tasks change as a result of fatigue. To this end, four raters were selected and each given 28 essays to score and comment on. Six general types of raters' comments (i.e., those on grammar, choice of words, organization, punctuation, dictation, and capitalization) were into focus in this study. Overall, results suggested that fatigue affects raters' frequency of comments on grammar, choice of words, and organization, and that raters' comments on punctuation, dictation, and capitalization do not seem to change significantly due to the effect of fatigue. Furthermore, this study revealed that the most and least frequent comments in 112 scored essays were those on grammar and dictation, respectively.

show abstract

Examination of Fatigue Effects From Extended‐time Accommodations on the Sat Reasoning Test™

Laitusis

Morgan

Bridgeman

et al. 2007

ETS Research Report Series

View full text Add to dashboard Cite

This study examined operational data from the SAT Reasoning Test™ to determine if students who tested under extended‐time conditions were suffering from excessive fatigue relative to students who tested under standard‐time conditions. Excessive fatigue was defined by significant (a) increases in differential item functioning (DIF) and (b) decreases in item completion rates, for items at the end of testing compared to the beginning of testing. Both of these factors were examined by comparing the performance of students who tested under standard time to students who tested with extended time on items administered early (section position 2 or 3) and different items administered late (section position 8, 9, or 10) during the 10‐section test administration. The sample included students with learning disabilities and/or Attention‐Deficit/Hyperactivity Disorder (ADHD) who tested with extended time (time and a half or double time) and students without disabilities who tested under standard‐time conditions. Analyses were conducted on the critical reading and writing sections of the SAT® and examined item difficulty as well as item completion rates. Results indicated few changes in levels of DIF (early in the test compared to late in the test). In addition, item completion rates for students who received extra time were comparable to (or in some cases higher than) test‐takers without disabilities who tested under standard time on both early and late sections.

show abstract

A Study of Fatigue Effects From the New Sat®

Cited by 11 publications

References 9 publications

Can Higher Education Exams Be Shortened? A Proposed Methodology

Can Higher Education Exams Be Shortened? A Proposed Methodology

Raters’ Fatigue and Their Comments During Scoring Writing Essays: A Case of Iranian Efl Learners

Examination of Fatigue Effects From Extended‐time Accommodations on the Sat Reasoning Test™

Contact Info

Product

Resources

About