Evaluating the rank‐ordering method for standard maintaining

Bramley, Tom; Gill, Tim

doi:10.1080/02671522.2010.498147

Cited by 13 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We anticipated that the peer assessment outcome would be consistent with the outcomes of the module summative tests. Third, because little is known about how judges make their decisions when comparing pairs of scripts (Bramley and Gill 2010), we investigated the possibility that peer judges might be making their judgements on the basis of surface features such as neatness and layout of work rather than on the underlying mathematical concepts. To do so, we recruited a group of novices: social science PhD students who had not studied mathematics beyond secondary school level, and in particular who in had never taken any courses in calculus.…”

Section: Research Focus and Designmentioning

confidence: 99%

Peer assessment without assessment criteria

Jones

Alcock

2013

Studies in Higher Education

110

115

View full text Add to dashboard Cite

Peer assessment typically requires students to judge peers' work against assessment criteria. We tested an alternative approach in which students judged pairs of scripts against one another in the absence of assessment criteria. First year mathematics undergraduates (N = 194) sat a written test of conceptual understanding of multivariable calculus, then assessed their peers' responses using pairwise comparative judgement. Inter-rater reliability was investigated by randomly assigning the students to two groups and correlating the two groups' assessments. Validity was investigated by correlating the peers' assessments with (i) expert assessments, (ii) novice assessments, and (iii) marks from other module tests. We found high validity and inter-rater reliability, suggesting that the students performed well as peer assessors. We interpret the results in the light of survey and interview feedback, and discuss directions for further research and development into the benefits and drawbacks of peer assessment without assessment criteria.

show abstract

Section: Research Focus and Designmentioning

confidence: 99%

Peer assessment without assessment criteria

Jones

Alcock

2013

Studies in Higher Education

110

115

View full text Add to dashboard Cite

show abstract

“…Yet feedback remains one of the major sources of improving the quality of course delivery and the provision of direct feedback to the teaching staff and other stakeholders (Keane and Labhraim, 2005). It is also known that academic service delivery over the span of two years is more likely to lead to improved quality if agreed standards are built into the practice of measurements (Bramley and Pollitt, 1998;Bramley and Gill, 2010).…”

Section: Conceptual Frameworkmentioning

confidence: 99%

Influence of Students’ Feedback on the Quality of Adult Higher Distance Education Service Delivery

Oduaran¹

2017

Turkish Online Journal of Distance Education

View full text Add to dashboard Cite

The evaluation of a program's compliance with service delivery and features necessary for the attainment of the programs educational objectives, student outcomes and continuous improvement is an important element in program accreditation and continuous improvement process. The study reported in this paper investigated the possible effects of students' feedback on the improvement of adult higher education distance learning service quality in a South African rural-based university. The study interrogated the service provision factors that seemingly helped in improving the delivery of the program. Such information are vital for planning, good governance, policy formulation, monitoring and evaluation, and for decision-making. The study used a quantitative descriptive statistics analysis of data generated ranging from 2013-2014. It comprised of overall student satisfaction as the dependent variable and the explanatory variables were given by program management, facilitation, assessment, learner support, systems, resources, program outcomes and subject matter. Analytical results were obtained from the Mann Whitney Test. The population consisted of students enrolled in the Advanced Certificate in Education (ACE) program by distance mode. The 313 respondents sampled were randomly selected from a total population of 916 students. Data were collected using a semi-structured questionnaire. The results revealed that service qualities linked to effective management, facilitation, academic support and subject matter delivery were the main qualities that the students recommended for the improvement of the program. It is therefore recommended that adult higher education programs must have a documented systematically utilized and effective process involving program service delivery constituencies like assessment, academic support and resources provided, for the periodic review of the program educational objectives to ensure that the program remains consistent with the institutional mission, needs and criteria.

show abstract

“…In addition, the use of collective expertise to construct a single rank order means the relative severity or leniency of individual experts is cancelled out and the statistical modelling allows the precision of the estimates to be quantified (Pollitt, ). Comparative judgement approaches prevail today as the methodology underpinning comparability studies in England and Wales (Bramley, ; Bramley & Gill, ).…”

Section: Measuring Standards Over Timementioning

confidence: 99%

Fifty years of A‐level mathematics: have standards changed?

Jones

Wheadon²,

Humphries

et al. 2016

British Educational Res J

View full text Add to dashboard Cite

OfqualAdvanced-level (A-level) mathematics is a high-profile qualification taken by many school leavers in England, Wales, Northern Ireland and around the world as preparation for university study. Concern has been expressed in these countries that standards in A-level mathematics have declined over time, and that school leavers enter university or the workplace lacking the required mathematical knowledge and skills. The situation in England, Wales and Northern Ireland reflects more general international concerns about decreasing educational standards. However, evidence to support this concern has been of limited scope, rarely subjected to peer-review and of questionable validity. Our study overcame the limitations of previous research into standards over time by applying a comparative judgement technique that enabled the direct comparison of mathematical performance across different examinations. Furthermore, unlike previous research, all examination questions were retypeset and candidate responses rewritten to reduce bias arising from surface cues. Using this technique, mathematics experts judged A-level scripts from the 1960s, 1990s and the 2010s. We report that the experts believed current A-level mathematics standards to have declined since the 1960s, although there was no evidence that they believed standards have declined since the 1990s. We contrast our findings with those from previous comparison studies and consider implications for future research into standards over time.

show abstract

Evaluating the rank‐ordering method for standard maintaining

Cited by 13 publications

References 9 publications

Peer assessment without assessment criteria

Peer assessment without assessment criteria

Influence of Students’ Feedback on the Quality of Adult Higher Distance Education Service Delivery

Fifty years of A‐level mathematics: have standards changed?

Contact Info

Product

Resources

About