Evaluation and Improvement of Intern Progress Note Assessments and Plans

Kelly, Michelle M.; Sklansky, Daniel J.; Nackers, Kirstin; Coller, Ryan J.; Dean, Shannon M.; Eickhoff, Jens; Bentley, Nicole L.; Nacht, Carrie L.; Shadman, Kristin A.

doi:10.1542/hpeds.2020-003244

Cited by 4 publications

(2 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…"Near-live" clinical simulation with "think-aloud" protocols can be deployed together with quantitative usability assessment like the System Usability Scale (SUS) [45][46][47] as part of a pragmatic usability test strategy [34] that prioritizes speed and cost-effectiveness. The SUS has been used extensively to assess the usability of the EHR [42,[48][49][50], including use to assess electronic note template usability [51]. The 10 statements on the SUS use a 5-point Likert scale to measure the strength of agreement or disagreement with each statement.…”

Section: Pragmatic Usability Testingmentioning

confidence: 99%

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: Prospective Improvement Study (Preprint)

Feldman¹,

Goodman²,

Hochman³

et al. 2022

Preprint

View full text Add to dashboard Cite

BACKGROUND The introduction of electronic workflows has allowed for the flow of raw un-contextualized clinical data into medical documentation.  As a result, many electronic notes have become replete of “noise” and deplete of clinically significant “signals”.  There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating.   OBJECTIVE To describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation.  METHODS This is a multi-hospital nonrandomized prospective improvement study conducted on the Inpatient General Internal Medicine Service across three hospital campuses at the New York University (NYU) Langone Health System.  A group of physician leaders representing each campus met biweekly for six months.  The output of these meetings included 1) a conceptualization of the note bloat problem as a dysfunction in information flow 2) a set of guiding principles for organizational documentation improvement 3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations and 4) a documentation improvement curriculum for inpatient medicine providers.  Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience measured using the System Usability Scale (SUS).  Primary outcomes measures after go-live include template utilization rate and note length in characters.   RESULTS In usability testing amongst 22 medicine providers, the new progress note template averaged a usability score of 90.6/100 on the System Usability Scale. 77% of providers strongly agreed that the new template was easy to use.  68% strongly agreed that they would like to use the template frequently.  In the three months after template implementation, General Internal Medicine providers wrote 65% of all inpatient notes with the new templates. During this period of time the organization saw a 46%, 47%, and 32% reduction in note length for general medicine progress notes, consults, and H&Ps, respectively, when compared to a baseline measurement period prior to interventions. CONCLUSIONS A bundled intervention that included deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency and patient outcomes.

show abstract

Section: Pragmatic Usability Testingmentioning

confidence: 99%

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: Prospective Improvement Study (Preprint)

Feldman¹,

Goodman²,

Hochman³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…“Near-live” clinical simulation with “think-aloud” protocols can be deployed together with quantitative usability assessment like the System Usability Scale (SUS) [ 45 - 47 ] as part of a pragmatic usability test strategy [ 34 ] that prioritizes speed and cost-effectiveness. The SUS has been used extensively to assess the usability of the EHR [ 42 , 48 - 50 ], including use to assess electronic note template usability [ 51 ]. The 10 statements on the SUS use a 5-point Likert scale to measure the strength of agreement or disagreement with each statement.…”

Section: Introductionmentioning

confidence: 99%

Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: Prospective Improvement Study

Feldman¹,

Goodman²,

Hochman³

et al. 2023

JMIR Form Res

View full text Add to dashboard Cite

Background The introduction of electronic workflows has allowed for the flow of raw uncontextualized clinical data into medical documentation. As a result, many electronic notes have become replete of “noise” and deplete clinically significant “signals.” There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating. Objective This study aims to describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation. Methods This is a multihospital nonrandomized prospective improvement study conducted on the inpatient general internal medicine service across 3 hospital campuses at the New York University Langone Health System. A group of physician leaders representing each campus met biweekly for 6 months. The output of these meetings included (1) a conceptualization of the note bloat problem as a dysfunction in information flow, (2) a set of guiding principles for organizational documentation improvement, (3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations, and (4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience was measured using the System Usability Scale (SUS). Primary outcome measures after go-live include template utilization rate and note length in characters. Results In usability testing among 22 medicine providers, the new progress note template averaged a usability score of 90.6 out of 100 on the SUS. A total of 77% (17/22) of providers strongly agreed that the new template was easy to use, and 64% (14/22) strongly agreed that they would like to use the template frequently. In the 3 months after template implementation, general internal medicine providers wrote 67% (51,431/76,647) of all inpatient notes with the new templates. During this period, the organization saw a 46% (2768/6191), 47% (3505/7819), and 32% (3427/11,226) reduction in note length for general medicine progress notes, consults, and history and physical notes, respectively, when compared to a baseline measurement period prior to interventions. Conclusions A bundled intervention that included the deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency, and patient outcomes.

show abstract