Reliability of large language models in managing odontogenic sinusitis clinical scenarios: a preliminary multidisciplinary evaluation

Saibene, Alberto Maria; Allevi, Fabiana; Calvo-Henriquez, Christian; Maniaci, Antonino; Mayo-Yáñez, Miguel; Paderno, Alberto; Vaira, Luigi Angelo; Felisati, Giovanni; Craig, John R.

doi:10.1007/s00405-023-08372-4

Cited by 8 publications

(1 citation statement)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some authors have even proposed investigating the potential benefits of artificial intelligence in decision-making for ODS scenarios. Saibene et al demonstrated the potential for AI to complement evidence-based clinical decision making, although there was still substantial disagreement between AI and clinical decisions, suggesting that AI is not yet optimal in assisting clinical management conclusions [61].…”

Section: Treatmentsmentioning

confidence: 99%

Pediatric Odontogenic Sinusitis: A Systematic Review

Rosso,

Urbanelli,

Spoldi

et al. 2024

JCM

Self Cite

View full text Add to dashboard Cite

Background: Pediatric odontogenic sinusitis (PODS) is a rare condition with limited research on its clinical features, diagnostic criteria, and treatment options. The current guidelines on pediatric rhinosinusitis do not mention a possible dental origin of the disease. This systematic review aims to summarize and analyze the existing literature on PODS, focusing on epidemiology, etiology, diagnostic tools, complications, treatment options, and outcomes. Methods: A systematic review was conducted following PRISMA reporting guidelines. Electronic searches were performed in multiple databases using keywords related to PODS and therapeutic strategies. Original articles reporting data on treatment outcomes for PODS were included. Results: The review highlighted the scarcity of high-quality evidence on PODS. The literature mainly consists of case reports and low-grade evidence studies. Limited data on the epidemiology, etiology, diagnostic tools, complications, and treatment outcomes of PODS in children are available. Conclusions: Further research is needed to better understand the clinical features, diagnosis, and treatment of PODS in pediatric patients. High-quality studies are required to establish evidence-based guidelines for the management of this condition, especially given the apparently high rate of complications when compared to adult ODS.

show abstract

Section: Treatmentsmentioning

confidence: 99%

Pediatric Odontogenic Sinusitis: A Systematic Review

Rosso,

Urbanelli,

Spoldi

et al. 2024

JCM

Self Cite

View full text Add to dashboard Cite

show abstract

Applications of ChatGPT in Otolaryngology–Head Neck Surgery: A State of the Art Review

Lechien,

Rameau

2024

Otolaryngol.--head neck surg.

View full text Add to dashboard Cite

ObjectiveTo review the current literature on the application, accuracy, and performance of Chatbot Generative Pre‐Trained Transformer (ChatGPT) in Otolaryngology–Head and Neck Surgery.Data SourcesPubMED, Cochrane Library, and Scopus.Review MethodsA comprehensive review of the literature on the applications of ChatGPT in otolaryngology was conducted according to Preferred Reporting Items for Systematic Reviews and Meta‐analyses statement.ConclusionsChatGPT provides imperfect patient information or general knowledge related to diseases found in Otolaryngology–Head and Neck Surgery. In clinical practice, despite suboptimal performance, studies reported that the model is more accurate in providing diagnoses, than in suggesting the most adequate additional examinations and treatments related to clinical vignettes or real clinical cases. ChatGPT has been used as an adjunct tool to improve scientific reports (referencing, spelling correction), to elaborate study protocols, or to take student or resident exams reporting several levels of accuracy. The stability of ChatGPT responses throughout repeated questions appeared high but many studies reported some hallucination events, particularly in providing scientific references.Implications for PracticeTo date, most applications of ChatGPT are limited in generating disease or treatment information, and in the improvement of the management of clinical cases. The lack of comparison of ChatGPT performance with other large language models is the main limitation of the current research. Its ability to analyze clinical images has not yet been investigated in otolaryngology although upper airway tract or ear images are an important step in the diagnosis of most common ear, nose, and throat conditions. This review may help otolaryngologists to conceive new applications in further research.

show abstract

Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms

Vaira,

Lechien,

Abbate

et al. 2024

Eur Arch Otorhinolaryngol

View full text Add to dashboard Cite

Background The widespread diffusion of Artificial Intelligence (AI) platforms is revolutionizing how health-related information is disseminated, thereby highlighting the need for tools to evaluate the quality of such information. This study aimed to propose and validate the Quality Assessment of Medical Artificial Intelligence (QAMAI), a tool specifically designed to assess the quality of health information provided by AI platforms. Methods The QAMAI tool has been developed by a panel of experts following guidelines for the development of new questionnaires. A total of 30 responses from ChatGPT4, addressing patient queries, theoretical questions, and clinical head and neck surgery scenarios were assessed by 27 reviewers from 25 academic centers worldwide. Construct validity, internal consistency, inter-rater and test–retest reliability were assessed to validate the tool. Results The validation was conducted on the basis of 792 assessments for the 30 responses given by ChatGPT4. The results of the exploratory factor analysis revealed a unidimensional structure of the QAMAI with a single factor comprising all the items that explained 51.1% of the variance with factor loadings ranging from 0.449 to 0.856. Overall internal consistency was high (Cronbach's alpha = 0.837). The Interclass Correlation Coefficient was 0.983 (95% CI 0.973–0.991; F (29,542) = 68.3; p < 0.001), indicating excellent reliability. Test–retest reliability analysis revealed a moderate-to-strong correlation with a Pearson’s coefficient of 0.876 (95% CI 0.859–0.891; p < 0.001). Conclusions The QAMAI tool demonstrated significant reliability and validity in assessing the quality of health information provided by AI platforms. Such a tool might become particularly important/useful for physicians as patients increasingly seek medical information on AI platforms.

show abstract

Reliability of large language models in managing odontogenic sinusitis clinical scenarios: a preliminary multidisciplinary evaluation

Cited by 8 publications

References 25 publications

Pediatric Odontogenic Sinusitis: A Systematic Review

Pediatric Odontogenic Sinusitis: A Systematic Review

Applications of ChatGPT in Otolaryngology–Head Neck Surgery: A State of the Art Review

Validation of the Quality Analysis of Medical Artificial Intelligence (QAMAI) tool: a new tool to assess the quality of health information provided by AI platforms

Contact Info

Product

Resources

About