The validity of student tutors’ judgments in early detection of struggling in medical school. A prospective cohort study

Abstract

Early identification and support of strugglers in medical education is generally recommended in the research literature, though very little evidence of the diagnostic qualities of early teacher judgments in medical education currently exists. The aim of this study was to examine the validity of early diagnosis of struggling in medical school based on informal teacher judgements of in-class behavior. The study design was a prospective cohort study and the outcomes/truth criteria were anatomy failure and medical school drop out. Six weeks into an anatomy course, student tutors attempted to identify medical students, who they reckoned would fail the anatomy course or drop out, based on their everyday experiences with students in a large group educational setting. In addition, they were asked to describe the indicators of struggling they observed. Sixteen student tutors evaluated 429 medical students for signs of struggling. By week six, the student tutors were able to detect approximately 1/4-1/3 of the students who eventually failed or dropped out, and for ¾ of the strugglers they identified, they were correct in their judgments. Informal student tutor's judgements showed incremental validity for both outcomes when controlling for grades obtained in preceeding exams. Lack of participation, lack of commitment, poor academic performance, poor social interactions and general signs of distress were the main indicators of struggling identified. Teachers' informal judgements of in-class behavior may be an untapped source of information in the early identification of struggling medical students with added value above and beyond formal testing.

Dissecting the role of sessional anatomy teachers: A systematic literature review

“…Interviews ( n = 2) and practical demonstrations of teaching skills ( n = 2) were the most commonly used selection tools, followed by reflective essays ( n = 1) and research proposals ( n = 1). Anatomical knowledge and/or academic merit was the most commonly cited selection criterion ( n = 6), with several studies indicating a preference for high‐performing near‐peer students (Durán et al, ; Erie et al, ; Lachman et al, ; Reyes‐Hernández et al, ; O'Neill et al, ). Enthusiasm for teaching was another key selection criterion ( n = 3) with one study stating that teachers were selected purely based on their “enthusiasm and willingness to teach” rather than their anatomical knowledge (Stephens et al, ).…”

Early identification of first-year students at risk of dropping out of high-school entry medical school: the usefulness of teachers’ ratings of class participation

“…The fact that an aggregate score of student participation in class collected systematically in a simple scale by multiple teachers can be useful in the process of identifying potential dropouts is, to our best knowledge, an original contribution. Prior studies had already shown the potential of using tutors' information to identify struggling students (Wijnia et al 2014;O'Neill et al 2016;Wijnia et al 2016), but in these studies the tutor/ students ratio was low and, therefore, there were more frequent contacts with the students. Our study provides other contextual circumstances.…”

“…Struggling might also be indicative of future lapses in professional behavior as a doctor, as suggested by retrospective studies in the USA and the UK (Papadakis et al 2004;Yates and James 2010). Signs of struggling identified by O'Neill et al (2016) include issues with participation (e.g. passivity, lack of initiative, lack of active participation in the classroom), commitment (e.g.…”

“…This study follows the suggestion by O'Neill et al (2016) to examine "a combination of teachers' independent informal judgments of specific in-class behavior (participation, commitment, academic performance, social interactions, distress) in addition to early performance testing" (p. 1076). We examined the relevance of including a mark computed from quantitative teachers' ratings of class participation as a predictor variable of dropouts.…”

Eighty Students, Five Cadavers, and Two Professors: Innovative Teaching Practices in an Undergraduate Human Gross Anatomy Course

“…If a student is struggling with material or not using effective study habits, the student can quickly fall behind to the point where they simply cannot be successful in the course. Instructor identification of struggling students can potentially be used to try to prevent this from occurring (O'Neill et al 2016). As a proactive approach, we use an informal in-class assessment in our scheduled lab periods to gauge student learning.…”