A Systematic Review of the Quality and Utility of Observer-Based Instruments for Assessing Medical Professionalism

Kwan, Yu Heng; Png, Kelly; Phang, Jie Kie; Leung, Ying Ying; Goh, Hendra; Seah, Yi; Thumboo, Julian; Ng, Pletneva; Fong, Warren; Lie, Désirée

doi:10.4300/jgme-d-18-00086.1

Cited by 23 publications

(23 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Moderators used a standardized guide (see supplementary materials) to identify issues related to medical professionalism that were important to patients. Our systematic review had earlier indicated that P-MEX was one of the promising tools to assess medical professionalism [25]. Therefore, we developed our topic guide based on the domains and items of the P-MEX, with an intent to adapt this tool for use in Singapore.…”

Section: Focus Group Discussion (Fgds) With Patientsmentioning

confidence: 99%

Assessment of medical professionalism: preliminary results of a qualitative study

et al. 2020

Self Cite

View full text Add to dashboard Cite

Background: The importance of medical professionalism has been well documented in the literature. Cultural background affects the constituents of professionalism. However, few studies have explored the domains of medical professionalism in an Asian context. We aim to describe the views and experiences of both patients and faculty on medical professionalism in an urban Asian city state. Methods: Data for this qualitative study were collected through focus group discussions (FGDs) with patients and in-depth interviews (IDIs) with faculty members. The IDIs and FGDs were audio-recorded and transcribed verbatim. Thematic analysis was conducted by two independent coders using a priori framework derived from Professionalism Mini Evaluation Exercise (P-MEX). Sociodemographic data of faculty members and patients were obtained through brief questionnaires. Results: A total of 31 patients (58.1% male, all had visited healthcare facilities within the past year) and 26 faculty members (42.3% male, 38.5% from medical disciplines, median years as faculty is 11) participated in this study. The results supported the four domains of medical professionalism -Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills. Two new sub-domains which emerged from data were (1) communicated effectively with patient and (2) demonstrated collegiality. Conclusions:The domains of professionalism in Singapore were similar to previous studies. This study allows a better understanding of medical professionalism to enhance the assessment and ultimately, the teaching of medical professionalism in an urban multi-ethnic Asian city.

show abstract

Section: Focus Group Discussion (Fgds) With Patientsmentioning

confidence: 99%

Assessment of medical professionalism: preliminary results of a qualitative study

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…For example, the item on maintaining appropriate boundaries received 100% agreement for inclusion into the assessment of medical professionalism. However, as the P-MEX was primarily designed to assess observable behaviours for use in the clinical setting, only 24 of the 142 behaviours were eventually chosen to be evaluated [17], and this has also resulted in the P-MEX having a higher utility score when used in the clinical setting to assess trainees [13]. Thus, for practical reasons, it will not be able to assess all domains of professionalism completely e.g.…”

Section: Discussionmentioning

confidence: 99%

“…To ensure strong retention of expert involvement, an upper limit of two rounds of investigation was set in this study [8]. It is also acknowledged that having a planned number of rounds is an indicator of good quality in designing a Delphi study [13].…”

Section: Designmentioning

confidence: 99%

Assessment of medical professionalism using the Professionalism Mini Evaluation Exercise (P-MEX) in a multi-ethnic society: a Delphi study

et al. 2020

Self Cite

View full text Add to dashboard Cite

Background: The importance of medical professionalism and its assessment has been well documented in the literature. However, there is currently no culturally-adapted tool to assess medical professionalism in Singapore. This study sets out to find consensus on relevance of the items from the Professionalism Mini Evaluation Exercise (P-MEX) for assessing medical professionalism in Singapore. Methods: A two-round Delphi survey was completed by an expert panel consisting of program directors, associate designated institutional officials, and designated institutional official (n = 15) from residency programs in Singapore. Round 1 comprised of 23 items from the P-MEX rated on a 5-point scale (1 = Definitely include, 2 = Possibly include, 3 = Neutral, 4 = Possibly exclude, 5 = Definitely exclude). In round 2, the experts received feedback from the previous round, and were asked to re-rate the items which did not achieve consensus in the previous round. The threshold for consensus in the study was defined as 70% or greater agreement among experts. Results: Completed questionnaires for both rounds were received from all 15 experts. In round 1, 18 items (78%) achieved consensus to be included. In round 2, 1 (4%) item achieved consensus to be included. However, none of the remaining items achieved consensus to be removed, and they exhibited stability in responses. A list of 19 items covering four domains of medical professionalism (Doctor-patient relationship skills, Reflective skills, Time management and Inter-professional relationship skills) was obtained during the two-rounds of Delphi survey. Conclusions: Nineteen items in the P-MEX had garnered consensus and is suitable for assessing medical professionalism in Singapore. The findings of this study can provide guidance for faculty and institutions who want to assess medical professionalism of their medical trainees.

show abstract

“…14 Despite these recommendations, a recent systematic review of such tools found that the one with the best psychometric properties has not yet been evaluated in either the US or in EM. 37 LaMantia et al recently developed a MSF tool that seems to have excellent internal consistency; however, its implementation was quite challenging and time intensive. 38 Given these limitations in the tools available, it is not surprising that this study demonstrates that some residencies simply provide faculty with the milestones and ask them to rate the residents.…”

Section: Discussionmentioning

confidence: 99%

Professionalism Milestones Assessments Used by Emergency Medicine Residency Programs: A Cross-sectional Survey

Stehman

Hochman

Fernández-Frackelton

et al. 2019

WestJEM

View full text Add to dashboard Cite

Introduction: Professionalism is a vital component of quality patient care. While competency in professionalism is Accreditation Council for Graduate Medical Education (ACGME)-mandated, the methods used to evaluate professionalism are not standardized, calling into question the validity of reported measurements. We aimed to determine the type and frequency of methods used by United States (US) -based emergency medicine (EM) residencies to assess accountability (Acc) and professional values (PV), as well as how often graduating residents achieve competency in these areas. Methods: We created a cross-sectional survey exploring assessment and perceived competency in Acc and PV, and then modified the survey for content and clarity through feedback from emergency physicians not involved in the study. The final survey was sent to the clinical competency committee (CCC) chair or program director (PD) of the 185 US-based ACGME-accredited EM residencies. We summarized results using descriptive statistics and Fisher’s exact testing. Results: A total of 121 programs (65.4%) completed the survey. The most frequently used methods of assessment were faculty shift evaluation (89.7%), CCC opinion (86.8%), and faculty summative evaluation (76.4%). Overall, 37% and 42% of residency programs stated that nearly all (greater than 95%) of their graduating residents achieve mastery of Acc and PV non-technical skills, respectively. Only 11.2% of respondents felt their programs were very effective at determining mastery of non-technical skills. Conclusion: EM residency programs relied heavily on faculty shift evaluations and summative opinions to determine resident competency in professionalism, with feedback from peers, administrators, and other staff less frequently incorporated. Few residency programs felt their current methods of evaluating professionalism were very effective.

show abstract

A Systematic Review of the Quality and Utility of Observer-Based Instruments for Assessing Medical Professionalism

Cited by 23 publications

References 35 publications

Assessment of medical professionalism: preliminary results of a qualitative study

Assessment of medical professionalism: preliminary results of a qualitative study

Assessment of medical professionalism using the Professionalism Mini Evaluation Exercise (P-MEX) in a multi-ethnic society: a Delphi study

Professionalism Milestones Assessments Used by Emergency Medicine Residency Programs: A Cross-sectional Survey

Contact Info

Product

Resources

About