Association Between Electronic Health Record Time and Quality of Care Metrics in Primary Care

Rotenstein, Lisa S; Holmgren, Anders; Healey, Michael; Horn, Daniel M; Ting, David; Lipsitz, Stuart; Salmasian, Hojjat; Gitomer, Richard S.; Bates, David W.

doi:10.1001/jamanetworkopen.2022.37086

Cited by 25 publications

(28 citation statements)

References 24 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, messaging is a critical resource for fostering patient equity, where individuals who have mobility limitations, work irregular hours, or fear medical bills, are potentially more likely to turn to messaging . High-quality responses might also improve patient outcomes . For some patients, responsive messaging may collaterally affect health behaviors, including medication adherence, compliance (eg, diet), and fewer missed appointments.…”

Section: Discussionmentioning

confidence: 99%

Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum

et al. 2023

View full text Add to dashboard Cite

ImportanceThe rapid expansion of virtual health care has caused a surge in patient messages concomitant with more work and burnout among health care professionals. Artificial intelligence (AI) assistants could potentially aid in creating answers to patient questions by drafting responses that could be reviewed by clinicians.ObjectiveTo evaluate the ability of an AI chatbot assistant (ChatGPT), released in November 2022, to provide quality and empathetic responses to patient questions.Design, Setting, and ParticipantsIn this cross-sectional study, a public and nonidentifiable database of questions from a public social media forum (Reddit’s r/AskDocs) was used to randomly draw 195 exchanges from October 2022 where a verified physician responded to a public question. Chatbot responses were generated by entering the original question into a fresh session (without prior questions having been asked in the session) on December 22 and 23, 2022. The original question along with anonymized and randomly ordered physician and chatbot responses were evaluated in triplicate by a team of licensed health care professionals. Evaluators chose “which response was better” and judged both “the quality of information provided” (very poor, poor, acceptable, good, or very good) and “the empathy or bedside manner provided” (not empathetic, slightly empathetic, moderately empathetic, empathetic, and very empathetic). Mean outcomes were ordered on a 1 to 5 scale and compared between chatbot and physicians.ResultsOf the 195 questions and responses, evaluators preferred chatbot responses to physician responses in 78.6% (95% CI, 75.0%-81.8%) of the 585 evaluations. Mean (IQR) physician responses were significantly shorter than chatbot responses (52 [17-62] words vs 211 [168-245] words; t = 25.4; P &lt; .001). Chatbot responses were rated of significantly higher quality than physician responses (t = 13.3; P &lt; .001). The proportion of responses rated as good or very good quality (≥ 4), for instance, was higher for chatbot than physicians (chatbot: 78.5%, 95% CI, 72.3%-84.1%; physicians: 22.1%, 95% CI, 16.4%-28.2%;). This amounted to 3.6 times higher prevalence of good or very good quality responses for the chatbot. Chatbot responses were also rated significantly more empathetic than physician responses (t = 18.9; P &lt; .001). The proportion of responses rated empathetic or very empathetic (≥4) was higher for chatbot than for physicians (physicians: 4.6%, 95% CI, 2.1%-7.7%; chatbot: 45.1%, 95% CI, 38.5%-51.8%; physicians: 4.6%, 95% CI, 2.1%-7.7%). This amounted to 9.8 times higher prevalence of empathetic or very empathetic responses for the chatbot.ConclusionsIn this cross-sectional study, a chatbot generated quality and empathetic responses to patient questions posed in an online forum. Further exploration of this technology is warranted in clinical settings, such as using chatbot to draft responses that physicians could then edit. Randomized trials could assess further if using AI assistants might improve responses, lower clinician burnout, and improve patient outcomes.

show abstract

Section: Discussionmentioning

confidence: 99%

Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum

et al. 2023

View full text Add to dashboard Cite

show abstract

“…This two-way interaction, between feeling burned out and better diagnosis documentation on one hand, and documentation burdens themselves contributing to burnout on the other, calls for more transformative thinking about the EHR and workflow redesign. 54 55 …”

Section: Discussionmentioning

confidence: 99%

“…The interaction between burnout and diagnosis documentation quality—as well as the vexing interaction between documentation itself and burnout. This two-way interaction, between feeling burned out and better diagnosis documentation on one hand, and documentation burdens themselves contributing to burnout on the other, calls for more transformative thinking about the EHR and workflow redesign 54 55…”

Section: Discussionmentioning

confidence: 99%

How does work environment relate to diagnostic quality? A prospective, mixed methods study in primary care

et al. 2023

View full text Add to dashboard Cite

ObjectivesThe quest to measure and improve diagnosis has proven challenging; new approaches are needed to better understand and measure key elements of the diagnostic process in clinical encounters. The aim of this study was to develop a tool assessing key elements of the diagnostic assessment process and apply it to a series of diagnostic encounters examining clinical notes and encounters’ recorded transcripts. Additionally, we aimed to correlate and contextualise these findings with measures of encounter time and physician burnout.DesignWe audio-recorded encounters, reviewed their transcripts and associated them with their clinical notes and findings were correlated with concurrent Mini Z Worklife measures and physician burnout.SettingThree primary urgent-care settings.ParticipantsWe conducted in-depth evaluations of 28 clinical encounters delivered by seven physicians.ResultsComparing encounter transcripts with clinical notes, in 24 of 28 (86%) there was high note/transcript concordance for the diagnostic elements on our tool. Reliably included elements were red flags (92% of notes/encounters), aetiologies (88%), likelihood/uncertainties (71%) and follow-up contingencies (71%), whereas psychosocial/contextual information (35%) and mentioning common pitfalls (7%) were often missing. In 22% of encounters, follow-up contingencies were in the note, but absent from the recorded encounter. There was a trend for higher burnout scores being associated with physicians less likely to address key diagnosis items, such as psychosocial history/context.ConclusionsA new tool shows promise as a means of assessing key elements of diagnostic quality in clinical encounters. Work conditions and physician reactions appear to correlate with diagnostic behaviours. Future research should continue to assess relationships between time pressure and diagnostic quality.

show abstract

“…Physicians with lower aFTE may have lower rates of burnout, so the higher EHR time per 8 scheduled hours among lower aFTE physicians may not translate into higher rates of burnout. Although it sacrifices nonclinical time, more EHR work outside scheduled hours for lower aFTE physicians may allow them to feel caught up or allocate more time for panel management work not afforded to physicians with higher aFTE …”

Section: Discussionmentioning

confidence: 99%

“…Although it sacrifices nonclinical time, more EHR work outside scheduled hours for lower aFTE physicians may allow them to feel caught up or allocate more time for panel management work not afforded to physicians with higher aFTE. 5 Study limitations include small sample size, including administrative EHR use unrelated to patient care that may preferentially be done by lower aFTE physicians, and being a single-center study. Given the increasing EHR workload and shortage of PCPs in the US, 6 future efforts should find ways to right-size workload for all aFTEs while allowing sufficient time for the inbox, proactive panel management, and quality improvement without affecting personal or nonclinical work time.…”

Section: Discussionmentioning

confidence: 99%

Association of Primary Care Physicians’ Ambulatory Full-time Equivalencies With Time in the Electronic Health Record

Micek

Rule

Baltus³

et al. 2023

JAMA Netw Open

View full text Add to dashboard Cite

show abstract

Association Between Electronic Health Record Time and Quality of Care Metrics in Primary Care

Cited by 25 publications

References 24 publications

Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum

Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum

How does work environment relate to diagnostic quality? A prospective, mixed methods study in primary care

Association of Primary Care Physicians’ Ambulatory Full-time Equivalencies With Time in the Electronic Health Record

Contact Info

Product

Resources

About