Consistency and Inconsistency in PhD Thesis Examination

Abstract

This is a mixed methods investigation of consistency in PhD examination. At its core is the quantification of the content and conceptual analysis of examiner reports for 804 Australian theses. First, the level of consistency between what examiners say in their reports and the recommendation they provide for a thesis is explored, followed by an examination of the degree of discrepancy between examiner recommendations and university committee decisions on the theses. Two groups of discrepant recommendations are identified and analysed in depth. Finally the main sources of inconsistency are identified. It was found that the comments of a small minority of examiners were inconsistent with each other or with the committee decision in a significant way. Much more commonly the texts of examiner reports were highly consistent and were closely reflected in the final committee decision.

Doctorates by thesis and publication in clinical medicine: An analysis of examiners’ reports

“…No study was identified that examined PhDs from Medicine exclusively, but no significant difference in length of reports between ten studied discipline categories, which included 'health', has been reported (Holbrook and Bourke, 2004). Another study of 804 Australian theses revealed that satisfied examiners wrote shorter reports than dissatisfied examiners (Holbrook et al, 2008). Holbrook et al (2004) note that examiners' perceptions of the value of a thesis were quantitative rather than qualitative; commendations were more often attached to theses thought to be good, and criticisms more commonly attached to poor theses.…”

Doctorateness across Higher Education Institutionsin Norway

“…In order to avoid these obstacles and delays, we believe that achievable deadlines for the completion process are necessary and should be adapted to the resources of the educational institutions, as well as outlined in their PhD regulations. Another point to be raised, is the lack of assessment criteria, which is a major concern in both national and international literature on doctorateness (Carter, 2008;Holbrook, 2008;Jones, 2013;Krumsvik et al, 2016;Morley, 2002).…”

Understanding the Value and the Impact of the ‘Shock’: Examining the Creative Writing Doctorate

“…Though it is a particular problem in creative arts disciplines, it is not peculiar to the arts; in any discipline, a candidate is likely to submit work that has a specific focus on one theoretical or epistemological framework, but also deploys particular methods and processes that may not be familiar to all potential examiners. It is not clear how examiners approach the problem of providing quality formative and summative commentary on a thesis where they are likely to have only partial expertise, and where they typically have had little if any training in the work of examination (Holbrook et al 2008). This is because examination is a process conducted largely in private, and it has attracted very little research (Carey, Webb, and Brien 2008).…”