Collaboration on progress testing in medical schools in the Netherlands

Schuwirth, Lambert; Bosman, G.J.C.G.M.; Henning, Robert H.; Rinkel, Rico N. P. M.; Wenink, Arnold C. G.

doi:10.3109/0142159x.2010.485658

Cited by 46 publications

(69 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Progress testing, quite popular in the Netherlands and other countries for several years, only recently has been valued as an evaluation tool in Brazilian educational institutions. Progress testing has been used by medical schools that have implemented curriculum changes, by some graduate programs and by some individual courses [1][2][3][4][5][6] .…”

Section: Introductionmentioning

confidence: 99%

“…The content of the test items is based on national curriculum guidelines for medical schools instituted by the Ministry of Education and Culture. The extant literature on medical education suggests that there are different ways of creating and implementing Progress testing [1][2][3][4][5][6]9 . A progress testing was applied twice a year to all students from 2001 to 2004 who were enrolled at the University of São Paulo School of Medicine.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

O Teste de Progresso como Indicador para Melhorias em Curso de Graduação em Medicina

Rosa

Isoppo

Cattaneo

et al. 2017

Rev. bras. educ. med.

View full text Add to dashboard Cite

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

O Teste de Progresso como Indicador para Melhorias em Curso de Graduação em Medicina

Rosa

Isoppo

Cattaneo

et al. 2017

Rev. bras. educ. med.

View full text Add to dashboard Cite

show abstract

“…The medical PT in the Netherlands is a collaboration between 5 of the 8 medical schools. The jointly constructed PT is taken by all medical students of the five medical schools at the same day and time four times a year (Schuwirth et al 2010). The benchmark group for the Students has a portfolio and the same mentor in all years of the program Four progress tests per year Feedback on progress test score/performance -total and by subject area The individual total score on each progress test is calculated (as described in the method section).…”

Section: Progress Testmentioning

confidence: 99%

Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment

et al. 2016

View full text Add to dashboard Cite

Background: Progress tests (PT) are used to assess students on topics from all medical disciplines. Progress testing is usually one of the assessment methods of the cognitive domain. There is limited knowledge on how positioning of the PT in a program of assessment (PoA) influences students' PT scores, use of PT feedback and perceived learning value. Methods: We compared PT total scores and use of a PT test feedback (ProF) system in two medical courses, where the PT is either used as a summative assessment or embedded in a comprehensive PoA and used formatively. In addition, an interview study was used to explore the students' perception on use of PT feedback and perceived learning value. Results: PT total scores were higher, with considerable effect sizes (ESs) and students made more use of ProF when the PT was embedded in a comprehensive PoA. Analysis of feedback in the portfolio stimulated students to look for patterns in PT results, link the PT to other assessment results, follow-up on learning objectives, and integrate the PT in their learning for the entire PoA. Conclusions: Embedding the PT in an assessment program designed according to the principles of programmatic assessment positively affects PT total scores, use of PT feedback, and perceived learning value.

show abstract

“…The item selection in the test is based on KL information. An illustrated example of the selection algorithm is given by building an item bank and performing CAPT simulations for the Dutch Medical Progress Test, which has been a standard paper-and-pencil test for many years (Schuwirth, Bosman, Henning, Rinkel, & Wenink, 2010).…”

Section: Journal Of Computerized Adaptive Testingmentioning

confidence: 99%

“…All first-to sixth-year students answer the same 200 items on various medical topics in a paper-and-pencil format. According to Schuwirth et al (2010), "the longitudinal feature of the progress test provides a unique and demonstrable measurement of the growth and effectiveness of students' knowledge acquisition." The goal of the test is to monitor the progress of students throughout the course of the medical program.…”

Section: The Dutch Medical Progress Testmentioning

confidence: 99%

Latent-Class-Based Item Selection for Computerized Adaptive Progress Tests

Buuren¹,

Eggen²

2017

JCAT

View full text Add to dashboard Cite

Standard computerized adaptive testing (CAT) methods require an underlying item response theory (IRT) model. An item bank can be constructed from the IRT model, and subsequent items can be selected with maximum information at the examinee's estimated ability level. IRT models, however, do not always fit test data exactly. In such situations, it is not possible to employ standard CAT methods without violating assumptions. To extend the scope of adaptive testing, this research shows how latent class analysis (LCA) can be used in item bank construction. In addition, the research investigates suitable item selection algorithms using KullbackLeibler (KL) information for item banks based on LCA. The KL information values can be used to select items and to construct an adaptive test. Simulations show that item selection based on KL information outperformed random selection of items in progress testing. The effectiveness of the selection algorithm is evaluated, and a possible scoring for the new adaptive item selection with two classes is proposed. The applicability of the methods is illustrated by constructing a computerized adaptive progress test (CAPT) on an example data set drawn from Misfit in IRT ModelsIn some situations, however, the standard IRT framework that underlies CAT does not fit the data, for example, during the calibration phase when building a CAT. Global fit of the IRT models to an item bank can be tested by looking at global fit measures such as the Q1 test, R1 test, and/or likelihood ratio tests (Andersen, 1973;Suárez-Falcón & Glas, 2003). These testing methods for global model fit can be first pointers to misfit. Concerns raised by global indications of misfit can be caused by violations of one or more assumptions of the IRT model. Item response functions can be flat instead of S-shaped, and problems with assumed local independence and unidimensionality can occur (Yang & Kao, 2014).If many items from the item bank show misfit, constructing an item bank might be impossible or only possible with heavy violations of the assumptions of standard IRT models. In these cases, other models can be applied to the data, such as multidimensional IRT models or latent variable mixture models. Multidimensional IRT can be modeled when multiple constructs in the data disturb the model fit (Béguin & Glas, 2001). Latent variable mixture modeling is useful in populations with heterogenous rather than homogenous samples (Sawatzky, Ratner, Kopec, Wu, & Zumbo, 2016). Sometimes, however, these other IRT models do not fit Research on LCA and CATThe present research appears to be the first to propose LCA in combination with CAT in an actual testing situation. Macready and Dayton (1992) investigated the use of LCA in CAT,concluding that the combination of CAT with LCA allows for conceptually simpler models than CAT with IRT. Macready and Dayton also acknowledged that LCA has fewer untestable assumptions. Their goal was to classify respondents into the appropriate latent class with CAT and to obtain an acceptable level of...

show abstract

Collaboration on progress testing in medical schools in the Netherlands

Cited by 46 publications

References 3 publications

O Teste de Progresso como Indicador para Melhorias em Curso de Graduação em Medicina

O Teste de Progresso como Indicador para Melhorias em Curso de Graduação em Medicina

Embedding of the progress test in an assessment program designed according to the principles of programmatic assessment

Latent-Class-Based Item Selection for Computerized Adaptive Progress Tests

Contact Info

Product

Resources

About