Why and When Physicians Google: Resident Physicians’ Information-Seeking Strategies During Patient Consultations

Hult, Helena Vallo; Östlund, Christian; Pålsson, Paul

doi:10.3233/shti230563

Cited by 2 publications

(1 citation statement)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is particularly crucial or threatening in elds such as medicine, when confused patients attempt to associate symptoms with diagnoses and therapies. Although these models are not primarily designed for this request, the practice will nd its way into people's everyday life in a similar way to googling symptoms by both patients [8,9] and physicians [10,11]. Moreover, preliminary studies have already demonstrated signi cant potential in utilizing ChatGPT within the medical eld.…”

Section: Introductionmentioning

confidence: 99%

Progression of Large Language Models for Clinical Decision Support: An Evaluation for Rare and Frequent Diseases using GPT-3.5, GPT 4 and Naïve Google Search

Varghese,

Sandmann,

Riepenhausen

et al. 2023

Preprint

View full text Add to dashboard Cite

Large Language Models (LLMs) like ChatGPT have become increasingly prevalent. Even without medical approval, people will use it to seek health advice, much like searching for diagnoses on Google. We performed a systematic analysis of GPT-3·5 and GPT-4 for suggesting diagnosis, examination steps and treatment of newly processed 110 medical case reports from different clinical disciplines. Balanced groups of rare, less frequent and frequent diseases were used as input. For the diagnosis task a naïve Google search was performed as benchmark comparison. Performance was assessed by two independent physicians using a 5-point Likert scale. The results showed superior performance of GPT-4 over GPT-3·5 considering diagnosis and examination and superior performance over Google for diagnosis. With the exception of treatment, better performance on frequent vs rare diseases was evident for all approaches. In conclusion, the LLMs showed growing potential for medical question answering in two successive major releases. However, several weaknesses and challenges necessitate the utilization of quality-controlled and regulated types of AI-models to qualify as medical applications.

show abstract

Section: Introductionmentioning

confidence: 99%

Progression of Large Language Models for Clinical Decision Support: An Evaluation for Rare and Frequent Diseases using GPT-3.5, GPT 4 and Naïve Google Search

Varghese,

Sandmann,

Riepenhausen

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

Exploring the impact of medical knowledge and information search skills on information searching behavior and quality of information found: A study comparing students-residents in medicine versus students in computer science (Preprint)

Chevalier,

Tamine,

Dosso

2024

Preprint

View full text Add to dashboard Cite

BACKGROUND Nowadays, Internet has become the primary source of information for physicians seeking answers to medical questions about their patients before consulting colleagues. However, many websites display information of low quality, less reliability, and lack scientific verification. Therefore, physicians need to develop information search skills to locate relevant and evidence-based information. However, previous studies have shown that physicians often have poor search skills and encounter difficulties in finding information on the Web. These challenges may have detrimental consequences on patient care. OBJECTIVE The present study aims to determine how medical students-residents search for medical information on Internet, what quality of web resources they utilize (including the nature and credibility of resources), and how they evaluate the reliability of web resources and the answers they provide. Given the importance of domain knowledge (in this case, medicine) and information search skills to the search activity, we compared the search behaviors of medical students and residents to those of students in computer science. Medical students-residents typically possess higher medical-related knowledge, while students in computer science generally have information search skills. METHODS Twenty students participated in this study: ten medical students-residents and ten students in computer science. Data were extracted from a freely accessible dataset. All participants were tasked with searching for information on the web to make a medical diagnosis, select a treatment, and enhance their knowledge of a medical condition—three primary activities they engage in online. We analyzed search performance metrics (such as search time, medical-related keywords used, and accuracy of information found) as well as the nature and credibility of web resources utilized by medical students and residents compared to students in computer science. RESULTS Students-residents in medicine provided, in mean, more accurate answers than students in computer science, all without requiring additional time. Their background in medicine also enabled them to better evaluate the reliability of resources and select high-quality web resources more effectively than students in computer science, primarily from hospital websites. However, it is noteworthy that students-residents in medicine utilized very few evidence-based tools, such as PubMed. CONCLUSIONS Although that students-residents in medicine had in general high performance than students in computer science, they did not use frequently evidence-based tools. As previously observed students-residents may avoid using databases due to the risk of encountering too many irrelevant articles and difficulties in applying correct filters to locate relevant information. Nevertheless, clinical and practical evidence-based medicine can significantly contribute to updating physicians' knowledge, improving patient care, and enhancing physician-patient relationships. Therefore, information search should be an integral part of medical training programs and continuing medical education for physicians.

show abstract

Why and When Physicians Google: Resident Physicians’ Information-Seeking Strategies During Patient Consultations

Cited by 2 publications

References 12 publications

Progression of Large Language Models for Clinical Decision Support: An Evaluation for Rare and Frequent Diseases using GPT-3.5, GPT 4 and Naïve Google Search

Progression of Large Language Models for Clinical Decision Support: An Evaluation for Rare and Frequent Diseases using GPT-3.5, GPT 4 and Naïve Google Search

Exploring the impact of medical knowledge and information search skills on information searching behavior and quality of information found: A study comparing students-residents in medicine versus students in computer science (Preprint)

Contact Info

Product

Resources

About