The questionnaire extension of the NGT provides back-up evidence of the reliability of the data derived from the technique and enables it to be applied to the larger groups typical of undergraduate medicine.

This long-term evaluation should examine whether existing assessments are congruent with the curriculum and relate to all facets of the students' learning experiences. This is particularly important in a curriculum where the learning outcomes of student-centred learning are emphasized. Changes in the assessment of postgraduate trainees and increasing emphasis on peer review of clinicians will raise the profile of these outcomes in undergraduate education.

This report describes the results of the project 'Assessment of Undergraduate Medical Education: Re-inventing the Wheel?' funded by the Nuffield Trust. The project was initiated in order to obtain information on current assessment practices in medical schools across the UK, to determine the extent of change as a consequence of the curricular reforms recommended by the General Medical Council (GMC) in 1993 and, if necessary, to stimulate debate about assessment and provide an impetus for change. The data obtained provide a detailed profile of the timing and nature of assessments used in medical schools and provide information for comparison with the survey of basic medical education in the British Isles carried out in 1977 and any subsequent studies. The study provides confirmatory and unexpected evidence. Schools are clearly revising their curricula consistent with recommendations made by the GMC. The main components of the study were as follows: a postal questionnaire sent to all UK medical schools, a national workshop, and four case studies of innovative assessment practices.

Absolute standard setting procedures are recommended for assessment in medical education. Absolute, test-centred standard setting procedures were introduced for written assessments in the Liverpool MBChB in 2001. The modified Angoff and Ebel methods have been used for short answer question-based and extended matching question-based papers, respectively. Data collected has been analysed to investigate whether reliable standards can be achieved for small-scale, medical school-based assessments, to establish the minimum number of judges required and the effect of a discussion phase on reliability. The root mean squared error (RMSE) has been used as a measure of reliability and used to compute 95% confidence intervals for comparison to the examination statistics. The RMSE has been used to calculate the minimum number of judges required to obtain a predetermined minimum level of reliability, and the effect of the number of judges and number of items have been examined. Values of the RMSE obtained vary from 0.9 to 2.2%. Using average variances across each paper type, the minimum number of judges to obtain a RMSE of less than 2% is 10 or more judges before discussion or 6 or more judges after discussion. The results indicate that including a discussion phase improves the reliability and reduces the minimum number of judges required. Decision studies indicate that increasing the number of questions included in the assessments would not significantly improve the reliability of the standard setting.

Summary Most medical schools in the UK are revising their undergraduate courses in response to the recommendations published by the General Medical Council Education Committee in Tomorrow's doctors. However, achievement of the objectives of curricular change is attendant on revision of the assessment process. This paper reviews traditional and more recently developed methods for assessment of medical education in the light of the General Medical Council's recommendations which relate specifically to summative assessment of the core curriculum. The importance of reliability and validity is highlighted, and the case for criterion-referenced assessment is examined.