Candidate Fatigue and Performance on Gce Objective Tests

Abstract

SUMMARY. Four multiple choice tests were set to groups of between 80 and 103 Advanced level chemistry students in their original form or with the items in reversed order. Analysis of the pattern of responses showed that the students seemed able to complete these tests in the time available. Analysis of covariance, using level of achievement (as measured by score in an A-level multiple choice test) as the covariate, of scores on items in original and reversed order forms, showed no evidence of differences in the proportions of students choosing the correct response which might be attributed to the effects of candidate fatigue or time limitations. INTRODUCTIONIt has long been recognised that students' performance on test questions may be affected by structural aspects of the test itself. Willmott and Hall (1975) have shown how, in tests involving question choice, performance declines on questions completed late in the test.Performance towards the end of the test may be inhibited by the onset of fatigue, by the effect of time limitations being most powerful late in the test, or by a combination of these. Whilst regarding these problems as significant in the context of non-objective tests Willmott and Hall did not investigate their effects in objective tests. They point out that most objective tests used in the General Certificate of Education (GCE) examinations are 'power' tests, completed by most candidates, rather than speeded tests where rate of working would have an effect on the scores obtained. Davidson and Carroll (1945) have shown that speed and power factors may be independent. Myers (1952) suggests that power may be more valid than speed as a predictor of academic

Raters’ Fatigue and Their Comments During Scoring Writing Essays: A Case of Iranian Efl Learners

“…In keeping with previous research studies (e.g., Massey, 1977;Wohlhueter, 1966;Weigle, 1994;McNamara, 1996;Hiramatsu, 2000;Liu et al, 2004;Goodall 2011;Sprouse, 2007;Drave, 2011;Ling et al, 2014), the present study highlighted the necessity of raters' training, and the importance of assessment protocols in order to avoid test bias. This study also suggested that fatigue can endanger even highly qualified raters' judgment in that the frequency and type of their comments on rated essays would change in an unfair manner from the first to the last few scored essays.…”

“…on raters' judgment (e.g., Bendig, 1955;Constable & Andrich, 1984;Cumming, Kantor, & Powers, 2002;Cumming, 1990;Weigle, 1994;McNamara & Wesley, 1996;Drave, 2011;Ling et al 2014;Lumley & McNamara, 1995;Massey, 1977;Schumm & Vaughn, 1991). Some researchers such as Weigle (1994), andMcNamara andWesley (1996) agree on the fact that with careful monitoring and training of raters, scoring procedure might end in reliable results and unbiased judgments.…”

“…Some researchers such as Weigle (1994), andMcNamara andWesley (1996) agree on the fact that with careful monitoring and training of raters, scoring procedure might end in reliable results and unbiased judgments. Too, some scholars (e.g., Cumming, 1954;Bendig, 1955;Drave, 2011;Tucker, 1948;Massey, 1977;Wohlhueter, 1966;Liu et al, 2004) maintain that construct-irrelevant factors, fatigue in particular, do not significantly affect test test-takers' scores and test-givers' scoring method. Bendig (1955), as a case in point, investigated the reliability of rater scoring and its possible loss as a result of fatigue and suggested that judgment fatigue did not affect scoring reliability.…”

“…Fatigue is one of the factors that can negatively affect human performance in general and scoring and commenting on essays, in particular. Although many studies have highlighted the effect of fatigue on test takers/givers' performance on language tests, (e.g., Bendig, 1955;Constable & Andrich 1984;Cumming et al, 2002;Cumming, 1990;Drave, 2011;Ling et al, 2014;Lumley & McNamara, 1995;Massey, 1977;Schumm & Vaughn, 1991), very little is known about its effects on a raters' scoring quality in speaking and writing tasks (Drave, 2011;Ling et al, 2014). Also, results of studies regarding the effect of fatigue in the literature were quite conflicting in the sense that some suggested that fatigue can affect human judgments' significantly in language tests (e.g., Tucker, 1948;Cumming, 1954;Bendig, 1955;Massey, 1977;Wohlhueter, 1966;Liu et al, 2004;Drave, 2011), whereas others argued that the effect of fatigue on test-takers', or raters' judgments is not significant (e.g., Wohlhueter, 1966;Hiramatsu, 2000;Goodall 2011;Sprouse, 2007).…”

Order Effects on Situational Judgment Test Items: A case of construct‐irrelevant difficulty

“…Mollenkopf () found evidence of increased item difficulty toward the end of a verbal test whereas Massey () found no evidence of an order effect. Monk and Stallings () found differences in average student performance between groups given one of two knowledge tests with randomly ordered items for 2 of 11 trials.…”