Open book tests: assessment of academic learning in clerkships

Abstract

An examination of an open-book testing approach in a family medicine clerkship seeks to determine whether this method more closely mirrors the discipline of family medicine, where practitioners refer daily to written resource materials in order to make clinical decisions without compromising the learning and assessment process. Student scores on the multiple-choice test were analysed by year, by quarter and by site using ANOVA. Students in the experimental site were interviewed to determine preparation style, use of text during test, as well as attitudes toward open-book testing. Analysis of variance showed that the interaction of site and year was significant at p = 0.03. The mean score of 88.2 for Maine students in 2002 was significantly different from the other three mean scores. The desired qualitative outcomes of the intervention were confirmed: reducing the anxiety of students, wider reading of the textbook, knowing the structure of the textbook as a learning resource, and deeper understanding of concepts and principles rather than time spent on memorization. While the difference in scores did reach statistical significance, it is important to note that the difference in mean score was only four points on a 100-point scale. Given the percentage of the total grade represented by the test score, it is unlikely that this difference represents any real difference in grade for students in Maine compared with Vermont. The students appeared to approach the textbook and therefore, perhaps, the body of knowledge as a whole with the orientation of a generalist. The MMC Clerkship Director recommended the implementation of the open-book approach to the Family Practice clerkship at all sites and the University of Vermont Medical School accepted the proposal. This recommendation supports advising students on the preparation for an open-book test and on tactics for the best use of the textbook during the test.

Taxation students’ perceptions of open-book assessment prior to the qualifying examination of South African chartered accountants

“…The focus group suggested that when preparing for open-book assessment, the layout of the textbook should be studied and arranged by using markers, flags and highlights. The importance of preparing the material in this way is confirmed by the findings by Broyles et al (2005). When it comes to the positive effects of the referencing habit, research has shown that maintaining a referencing habit while preparing for an open-book assessment is essential, but during an open-book assessment it should be used sparingly, and only as a back-up (Boniface, 1985;Jacobs, 1990).…”

“…Broyles et al (2005) concur by stating that being aware of time is vitally important. Time plays a role during preparation, while writing the assessment and when using the textbooks (tools) available during the assessment.…”

“…The focus group maintained that, a student who had access to information could score extra marks, but, on the other hand, the student could also include irrelevant information in the solution. Broyles et al (2005) concurred when they advised against the use of available materials to confirm every answer, and advised that this should be done only when necessary.…”

“…This leads to the conclusion that in most of the projects, the full potential of open-book assessment was not used, for example, allowing students to apply their knowledge creatively (Feller 1994). This still tends to be the case, although in more and more research on open-book assessment it is regarded as a tool and technique for simulating real-life situations (Broyles, Cyr & Korsen, 2005 Currently, open-book assessment is seen as one of the alternative assessment types that include assessment requiring students to generate their own responses rather than choosing or taking a response from a given list or text (Jackson, Watty, Yu & Lowe, 2006). Open-book assessment allows students to make use of different materials during the assessment process (Theophilides & Koutselini-Ioannidou, 2000).…”

“…More specifically, the work of Theophilides and Koutselini-Ioannidou (2000) found that examination anxiety and feelings of pressure seemed to decrease during openbook assessment. (Boniface, 1985;Broyles et al, 2005;Feldhusen, 1961;Jacobs, 1990). From the didactical perspective, numerous researchers have concluded that, when students understand the content, proficiency and professional skills are more easily attained (Brightwell et al, 2004;Feldhusen, 1961;Gray, 1993;Jacobs, 1990).…”