Computer-based oral exams in Italian language studies

Abstract

In this paper we report on one component of a three-year study into the use of digital technologies for summative performance assessment in senior secondary courses in Western Australia. One of the courses was Italian Studies, which had an oral communication outcome externally assessed with an oral performance for which students travelled to a central location and undertook an interview with two assessors. Apart from the logistical difficulties for both students and the organising body, this method didn't leave an enduring record of the process, and raised questions about the reliability of the assessment. Over the three years of this study, we tried several approaches to using digital technology to assess oral performance, including a portfolio of sub-tasks leading up to a video-recorded oral presentation, a computer-based exam, a video recorded interview, and an online exam that included oral audio-recordings. For each of the years online marking tools supported two methods of drawing inferences about student performance from the representations: the more traditional analytical method and the comparative pairs method. Rasch analysis of the results of the two methods showed that both were at an acceptable level of reliability. Overall students and teachers reported that they liked using audiovisual recordings and online performance tasks for revision but not for summative assessment. The study also demonstrated that the scores from externally marked computer-based oral tasks carried out in class time correlated highly with the scores from traditional face-to-face recorded interviews. Therefore, online assessment of oral performance appears to be an equally effective way to facilitate assessment when compared with traditional methods and offers other affordances, such as convenience and access from a variety of locations, as well as providing an enduring record of student output.

Computer Adaptive Language Testing According to Nato Stanag 6001 Requirements

“…Thus, CAT has become a particularly wide application for solving educational problems (Weiss & Kingsbury, 1984;Maia, Lilley & Barker, 2003;Babcock & Weiss, 2012;Oppl et al, 2017). The scientists (Blake, 2011;Hambleton,& Zaal;Newhouse & Cooper, 2013) study three main types of the computer adaptive testing: in pyramidical testing the test taker first gets the items of medium complexity, and then depending on his/her answers, easier or more complex items are provided; (Larkin & Weiss, 1975); flexilevel testing starts with the complexity level chosen by a test taker, and then depending on his/her answers, the next item is easier or more complex than the previous one; it goes that way until the test taker's knowledge level is defined (Lord, 1971); stradaptive testing is held with the help of the bank of items divided according to complexity levels. If a test taker gives the correct answer to the item, his/her next item is suggested at a higher level of complexity and vice versa (Weiss, 1973, р.…”

CALL research: Where are we now?

“…Such honesty is positive, as is the willingness to report a project failing to produce the hoped-for results, but tends to undermine their impact. The evidential value of these often excellent studies could be enhanced if they were broad in approach, conducted over an extended period, with a range of students, including all the students of a cohort, often with other institutions, and collecting a significant amount of data, from a range of sources; for example, projects such as Newhouse and Cooper's (2013) work on computer-based oral exams in Italian language studies. This pattern is consistent across all three journals.…”

Computerized testing in reading comprehension skill: investigating score interchangeability, item review, age and gender stereotypes, ICT literacy and computer attitudes

“…When the same or similar test is implemented in its alternative mode, and received scores demonstrate that test takers show the same level of proficiency, then scores are considered reliable. The alternative versions of tests should produce sustainable valid and reliable measures of intended proficiency (Newhouse & Cooper, 2013 ). According to the guidelines published by American Educational Research Association (AERA), if more than one way of different ways of implementing a test is used, scores received from the ways should be interchangeable (AERA, 2014 ).…”