On the Meaning of Scores Derived from Patient Management Problems

Abstract

Patient Management Problem (PMP) test scores have been questioned by many for their interpretability and validity. Despite these measurement problems, PMPs remain popular evaluation and research instruments. A paired-comparisons study was undertaken to investigate the relationship between performance and corresponding scores (proficiency andpathway). The scale values for the performances do not reflect the same relative quality ofperformance as the proficiency and pathway scores. Even more remarkable is the emergence of a different standard when performance is assessed as a whole. The findings of this study suggest that assigning points at the "option level" culminates in scores that cannot easily be interpreted and cannot readily be linked to the behavior they are supposed to represent. These conclusions have implications for construct validity and educational decisions.