Teachers’ opinions on quality criteria for Competency Assessment Programs

Abstract

Quality control policies towards Dutch vocational schools have changed dramatically because the government questioned examination quality. Schools must now demonstrate assessment quality to a new Examination Quality Center. Since teachers often design assessments, they must be involved in quality issues. This study therefore explores teachers' opinions on assessment quality evaluation criteria. Pre-vocational and vocational teachers (N ¼ 211) responded to a questionnaire. Contrary to expectations, results show that teachers deem classical and competency-based quality criteria equally important. Vocational teachers gave higher importance scores than pre-vocational teachers, possibly due to the pressure they experience to improve the quality of their assessments. r 2006 Elsevier Ltd. All rights reserved. There is a strong pressure on educational institutes and teachers to become more competency-based, so as to better meet the changing demands of the labor market. This has important consequences for student assessment because of the strong relationship that exists between instruction, learning and assessment. Assessment, learning and instruction should be aligned with each other (i.e. focus on the same learning outcomes). Also, assessment appears to strongly influence both how students learn and how teachers teach, causing both students and teachers to focus on what the assessment requires (e.g. Alderson & Wall, 1993;Biggs, 1999;Birenbaum, 2003;Frederiksen, 1984). A study focusing on how teachers connect instruction and assessment showed that teachers spend more than 35% of their time on assessment and more than 10% on assessment-driven instruction (Conca, Schechter, & Castle, 2004). A possible problem here is that while learning and instruction are increasingly competency-based, the development of adequate methods to assess those competencies appears to be lagging behind. The past decade has seen a number of new assessment forms, such as performance assessment, authentic ARTICLE IN PRESS www.elsevier.com/locate/tate 0742-051X/$ -see front matter r

Validity as a social imperative for assessment in health professions education: a concept analysis

“…There has been increasing desire on the part of society for information on how professional bodies and higher education institutions measure and monitor the acquisition of competencies by health care professionals . For example, one of the core tenets of competency‐based education (CBE) is to protect patients by ensuring that professionals achieve a certain level of competency and are better prepared for the realities of the clinical setting . Professional bodies and higher education institutions must ensure that their assessment programmes are carefully planned and designed in order to measure the abilities of learners across targeted competencies as accurately as possible.…”

“…Messick's 13 unified theory of validity has yet to be operationalised in a way that could help those responsible for the documentation of the quality of their assessment programmes. 25,47 Although there have been efforts to operationalise Kane's 14,43 argument-based approach to the validation of inferences based on assessment scores for programmatic assessment, this framework does not have the same focus on consequences of assessment as that of Messick. 13 Further work is required to bring the best of these frameworks together in the current context of HPE assessment in order to support validation practices for programmes of assessment.…”

Kutseõppedidaktika teoorias ja praktikas: üliõpilaste tähelepanekud vaatluspraktikal

“…Õpetaja on oma pädevustega õppijatele kutsealal eeskujuks. (de Bruijn & Leeman, 2011) Hindamisel võidakse keskenduda kas eraldi teadmistele ja oskustele või tervik likele kompetentsidele (Baartman, Bastiaens, Kirschner, & van der Vleuten, 2007;de Bruijn & Leeman, 2011). Kutseõppes rakendatakse väljundipõhist hindamist, lähtudes õppekavas kirjeldatud õpiväljundite saavutatusest ning tagades seeläbi kindlate hindamiskriteeriumite abil õiglase, autentse ja erapooletu hinnangu õppija teadmiste ja oskuste omandamise taseme kohta (Kutse õppes kasutatav ..., Noorväli et al, 2014).…”

“…(Ingle & Duckworth, 2013;Kutseõppes kasutatav ..., 2013). Teoreetiliste teadmiste või töösoorituse kvaliteedi hindamiseks on kasutusel erinevad kompetentside omandatuse tõenda mise viisid ehk hindamismeetodid (test, esitlus, projekt, rühmatöö, praktiline töö, autentses situatsioonis oskuste demonstratsioon vms) ning hindamis tüübid (Baartman et al, 2007;de Bruijn & Leeman, 2011). Seega võib hindamine toimuda õppija enesehindamisena, kaasõppijate vastastikhindamisena ja/või juhendaja juhituna, kui juhendaja annab hinnangu õppija kompetentside tasemele.…”

Constraints in Teacher Training for Computer Assisted Language Testing Implementation

“…Besides, although most students of translation, language education and philology in Spain (and possibly in the rest of Europe) will probably spend many years (if not to say their whole careers) in teaching --in which assessment is a crucial part of their job--, they receive almost no preparation for assessment in their university lessons (Stiggins & Conklin, 2002). Thus, teachers are not aware of the different factors involved in language testing (Cheng et al, 2008), and at the same time they need to be trained in this area (Baartman et al, 2007) as well as in the implications of high-stakes testing beyond their students' need to obtain the highest score possible (Luxia, 2007). Consequently, in this "post-methodology" era (Kumaravadivelu, 2002), many instructors tend to use a limited number of tasks which may go against the idea of communicative use of language in education as both a means and a target (Savignon, 2007), even when doing so may go against their own beliefs (Saif, 2006;Turner, 2006) and thus lead to negative washback.…”