Thirteen experienced supervising psychiatrists independently rated the psychotherapy interviews of six residents, using a detailed assessment instrument. Although interrater agreement was significantly greater than would be expected by chance, the extent of agreement was uniformly low. Agreement was significantly poorer for judgments about technique skills than about communication skills and for second-year than for first-year residents. Other factors, such as the faculty rank and training of the rater, did not covary significantly with observed ratings. The findings suggest that rating methods which purport to assess psychotherapy skills should undergo careful methodological evaluation and testing before they are accepted as valid assessment techniques.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.