Developing Validity Evidence for the Written Pediatric History and Physical Exam Evaluation Rubric

King, Marta; Phillipi, Carrie A.; Buchanan, Paula; Lewin, Linda

doi:10.1016/j.acap.2016.08.001

Cited by 7 publications

(29 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students' documentation of pediatric history and physical notes. 15 These data are markedly different from the data in previous studies that have found low interrater reliability for psychometric evaluations related to judgment and decision-making. [36][37][38][39] CRANAPL was also found to have high intrarater reliability, which shows the reproducibility of an individual's assessment over time.…”

Section: Discussioncontrasting

confidence: 80%

“…Prior studies have described rubrics for evaluating the clinical reasoning skills of medical students. 14,15 The ICCs for the IDEA rubric used to assess medical students' documentation of clinical reasoning were fair to moderate (0.29-0.67), whereas the ICC for the CRANAPL tool was high at 0.83. This measure of reliability is similar to that for the P-HAPEE rubric used to assess medical students' documentation of pediatric history and physical notes.…”

Section: Discussionmentioning

confidence: 99%

“…The strong association between the total CRANAPL score and global clinical reasoning assessment found in the present study is similar to that found in previous studies that have also embedded global rating scales as comparators when assessing clinical reasoning. 13,15,40,41 Global rating scales represent an overarching structure for comparison given the absence of an accepted method or gold standard for assessing clinical reasoning documentation. High-quality provider notes are defined by clarity, thoroughness, and accuracy; 35 and effective documentation promotes communication and the coordination of care among the members of the care team.…”

Section: Discussionmentioning

confidence: 99%

“…[23][24][25][26] For each of these parties, diagnostic reasoning and its impact on clinical decision-making are considered to be a core competency. Several works that heavily influenced the CRANAPL tool's development were Baker's Interpretive Summary, Differential Diagnosis, Explanation of Reasoning, And Alternatives (IDEA) assessment tool; 14 King's Pediatric History and Physical Exam Evaluation (P-HAPEE) rubric; 15 and three other studies related to diagnostic reasoning. 16,27,28 These manuscripts and other works substantively informed the preliminary behavioral-based anchors that formed the initial foundation for the tool under development.…”

Section: Content Validitymentioning

confidence: 99%

“…11,12 Studies that have formally evaluated the documentation of clinical reasoning have focused exclusively on medical students. [13][14][15] The nonexistence of a detailed rubric for evaluating clinical reasoning in the A&Ps of hospitalists represents a missed opportunity for evaluating what hospitalists "do"; if this evolves into a mechanism for offering formative feedback, such profes-…”

mentioning

confidence: 99%

See 4 more Smart Citations

Documentation of Clinical Reasoning in Admission Notes of Hospitalists: Validation of the CRANAPL Assessment Rubric

Kotwal

Klimpl

Tackett

et al. 2019

Journal of Hospital Medicine

View full text Add to dashboard Cite

OBJECTIVE: To establish a metric for evaluating hospitalists’ documentation of clinical reasoning in admission notes. STUDY DESIGN: Retrospective study. SETTING: Admissions from 2014 to 2017 at three hospitals in Maryland. PARTICIPANTS: Hospitalist physicians. MEASUREMENTS: A subset of patients admitted with fever, syncope/dizziness, or abdominal pain were randomly selected. The nine-item Clinical Reasoning in Admission Note Assessment & Plan (CRANAPL) tool was developed to assess the comprehensiveness of clinical reasoning documented in the assessment and plans (A&Ps) of admission notes. Two authors scored all A&Ps by using this tool. A&Ps with global clinical reasoning and global readability/clarity measures were also scored. All data were deidentified prior to scoring. RESULTS: The 285 admission notes that were evaluated were authored by 120 hospitalists. The mean total CRANAPL score given by both raters was 6.4 (SD 2.2). The intraclass correlation measuring interrater reliability for the total CRANAPL score was 0.83 (95% CI, 0.76-0.87). Associations between the CRANAPL total score and global clinical reasoning score and global readability/clarity measures were statistically significant (P < .001). Notes from academic hospitals had higher CRANAPL scores (7.4 [SD 2.0] and 6.6 [SD 2.1]) than those from the community hospital (5.2 [SD 1.9]), P < .001. CONCLUSIONS: This study represents the first step to characterizing clinical reasoning documentation in hospital medicine. With some validity evidence established for the CRANAPL tool, it may be possible to assess the documentation of clinical reasoning by hospitalists.

show abstract

Section: Discussioncontrasting

confidence: 80%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Content Validitymentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Documentation of Clinical Reasoning in Admission Notes of Hospitalists: Validation of the CRANAPL Assessment Rubric

Kotwal

Klimpl

Tackett

et al. 2019

Journal of Hospital Medicine

View full text Add to dashboard Cite

show abstract

Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback

et al. 2021

View full text Add to dashboard Cite

BACKGROUND: Residents and fellows receive little feedback on their clinical reasoning documentation. Barriers include lack of a shared mental model and variability in the reliability and validity of existing assessment tools. Of the existing tools, the IDEA assessment tool includes a robust assessment of clinical reasoning documentation focusing on four elements (interpretive summary, differential diagnosis, explanation of reasoning for lead and alternative diagnoses) but lacks descriptive anchors threatening its reliability. OBJECTIVE: Our goal was to develop a valid and reliable assessment tool for clinical reasoning documentation building off the IDEA assessment tool. DESIGN, PARTICIPANTS, AND MAIN MEASURES: The Revised-IDEA assessment tool was developed by four clinician educators through iterative review of admission notes written by medicine residents and fellows and subsequently piloted with additional faculty to ensure response process validity. A random sample of 252 notes from July 2014 to June 2017 written by 30 trainees across several chief complaints was rated. Three raters rated 20% of the notes to demonstrate internal structure validity. A quality cut-off score was determined using Hofstee standard setting. KEY RESULTS: The Revised-IDEA assessment tool includes the same four domains as the IDEA assessment tool with more detailed descriptive prompts, new Likert scale anchors, and a score range of 0-10. Intraclass correlation was high for the notes rated by three raters, 0.84 (95% CI 0.74-0.90). Scores ≥6 were determined to demonstrate high-quality clinical reasoning documentation. Only 53% of notes (134/252) were high-quality. CONCLUSIONS: The Revised-IDEA assessment tool is reliable and easy to use for feedback on clinical reasoning documentation in resident and fellow admission notes with descriptive anchors that facilitate a shared mental model for feedback.

show abstract

Development and Validation of a Machine Learning Model for Automated Assessment of Resident Clinical Reasoning Documentation

et al. 2022

View full text Add to dashboard Cite

Developing Validity Evidence for the Written Pediatric History and Physical Exam Evaluation Rubric

Cited by 7 publications

References 22 publications

Documentation of Clinical Reasoning in Admission Notes of Hospitalists: Validation of the CRANAPL Assessment Rubric

Documentation of Clinical Reasoning in Admission Notes of Hospitalists: Validation of the CRANAPL Assessment Rubric

Development of a Clinical Reasoning Documentation Assessment Tool for Resident and Fellow Admission Notes: a Shared Mental Model for Feedback

Development and Validation of a Machine Learning Model for Automated Assessment of Resident Clinical Reasoning Documentation

Contact Info

Product

Resources

About