An Extensive Comparative Analysis of Chatbot Technologies - ChatGPT, Google BARD and Microsoft Bing

Bhardwaz, Saumyamani; Kumar, Jitender

doi:10.1109/icaaic56838.2023.10140214

Cited by 22 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The data used for training these AI models are different, which can explain the variations in their responses. These variations can lead to differences in the narratives of the chatbots when providing human‐like responses (Bhardwaz & Kumar, 2023; Cascella et al., 2023). GPT‐3.5 is based on GPT, which is designed to generate human‐like text by predicting the likelihood of a word given the preceding words in a sentence.…”

Section: Discussionmentioning

confidence: 99%

“…On the other hand, while it is known that Google Bard is based on BERT (Pichai, 2023) and Bing is based on GPT (Mehdi, 2023) technologies, the exact architectures and technical details about the models are not available publicly, which limits the ability to understand why the chatbots perform so differently when endodontic questions were asked. The differences in chatbot responses can also be attributed to the varying design philosophies of the AI companies, the specific algorithms employed, the datasets used for training, and the objectives that the AI is designed to achieve (Bhardwaz & Kumar, 2023; Jianfeng et al., 2019).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics

Mohammad‐Rahimi,

Ourang,

Pourhoseingholi

et al. 2023

Int Endodontic J

View full text Add to dashboard Cite

AimThis study aimed to evaluate and compare the validity and reliability of responses provided by GPT‐3.5, Google Bard, and Bing to frequently asked questions (FAQs) in the field of endodontics.MethodologyFAQs were formulated by expert endodontists (n = 10) and collected through GPT‐3.5 queries (n = 10), with every question posed to each chatbot three times. Responses (N = 180) were independently evaluated by two board‐certified endodontists using a modified Global Quality Score (GQS) on a 5‐point Likert scale (5: strongly agree; 4: agree; 3: neutral; 2: disagree; 1: strongly disagree). Disagreements on scoring were resolved through evidence‐based discussions. The validity of responses was analysed by categorizing scores into valid or invalid at two thresholds: The low threshold was set at score ≥4 for all three responses whilst the high threshold was set at score 5 for all three responses. Fisher's exact test was conducted to compare the validity of responses between chatbots. Cronbach's alpha was calculated to assess the reliability by assessing the consistency of repeated responses for each chatbot.ResultsAll three chatbots provided answers to all questions. Using the low‐threshold validity test (GPT‐3.5: 95%; Google Bard: 85%; Bing: 75%), there was no significant difference between the platforms (p > .05). When using the high‐threshold validity test, the chatbot scores were substantially lower (GPT‐3.5: 60%; Google Bard: 15%; Bing: 15%). The validity of GPT‐3.5 responses was significantly higher than Google Bard and Bing (p = .008). All three chatbots achieved an acceptable level of reliability (Cronbach's alpha >0.7).ConclusionsGPT‐3.5 provided more credible information on topics related to endodontics compared to Google Bard and Bing.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics

Mohammad‐Rahimi,

Ourang,

Pourhoseingholi

et al. 2023

Int Endodontic J

View full text Add to dashboard Cite

show abstract

“…The selection of these AI models was strategic, aiming to leverage their distinct capabilities and areas of expertise. ChatGPT 4 and Microsoft Copilot were chosen for their proven track record and extensive documentation in scientific literature, which has established them as reliable tools for complex data interpretation [6,14,15]. On the other hand, the inclusion of the Gemini and Gemini Advanced models was motivated by their novelty and the claims of their developers regarding their advanced analytical capabilities [16].…”

Section: Application Of the Ai Modelmentioning

confidence: 99%

AI-Powered Western Blot Interpretation: A Novel Approach to Studying the Frameshift Mutant of Ubiquitin B (UBB+1) in Schizophrenia

Fabijan,

Chojnacki,

Zawadzka-Fabijan

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

The application of artificial intelligence (AI) in the analysis of molecular biology data is becoming increasingly widespread. The Western Blot (WB) technique, a cornerstone in proteomic research, facilitates the identification and analysis of proteins, such as the frameshift mutant of ubiquitin B (UBB+1). In our study, we attempted to assess the potential of four different AI models—Gemini, Gemini Advanced, Microsoft Copilot, and ChatGPT 4—in the analysis of WB imagery containing UBB+1, derived from peripheral blood studies of patients suffering from schizophrenia. Participants, all male and diagnosed with schizophrenia, were recruited from the Specialist Psychiatric Care Team of Babinski Hospital in Lodz. After obtaining their informed consent, blood samples were collected and transported to the laboratory of the Department of Medical Biochemistry at the Medical University of Lodz. The samples were processed, synthesis of Ub-48UBB+1 dimers was performed, and the WB technique was applied. The result of the WB analysis, in the form of a photograph with basic labels but without a legend (JPG format), was implemented into ChatGPT 4, Microsoft Copilot, Gemini and Gemini Advanced. Following the implementation of the image, the command ‘Could you analyze the attached photo?’ was added, along with the protocol from Sample Preparation and Synthesis of Ub-48UBB+1 Dimers. The AI models effectively analyzed and interpreted WB images, with variations in their approaches and depth. Gemini excelled in detailing the WB process and biological significance of bands, while Gemini Advanced focused on specific band identification, especially Ub-48UBB+1 dimers. Microsoft Copilot provided a basic overview with less technicality, and ChatGPT 4 offered comprehensive band interpretations, linking them to patient samples and standards, thus confirming the hypothesis about the differing capabilities of these models. This discovery demonstrates the advanced capabilities of ChatGPT 4 and highlights the growing role of AI in scientific research, including the interpretation of results.

show abstract

“…Mainly used in this study are ChatGPT (https://openai.com/blog/chatgpt) (accessed 2 April 2024) and Gemini (https://gemini. google.com/app) (accessed 3 April 2024), which exhibit remarkable capabilities in processing and generating human-like text across diverse domains [1,2]. ChatGPT is based on the Generative Pre-Trained (GPT) architecture, including GPT-3 and GPT-4.…”

Section: Introductionmentioning

confidence: 99%

AI for Biophysical Phenomena: A Comparative Study of ChatGPT and Gemini in Explaining Liquid–Liquid Phase Separation

Rana,

Katoch

2024

Applied Sciences

View full text Add to dashboard Cite

Recent advancements in artificial intelligence (AI), notably through generative pretrained transformers, such as ChatGPT and Google’s Gemini, have broadened the scope of research across various domains. Particularly, the role of AI in understanding complex biophysical phenomena like liquid–liquid phase separation (LLPS) is promising yet underexplored. In this study, we focus on assessing the application of these AI chatbots in understating LLPS by conducting various interactive sessions. We evaluated their performance based on the accuracy, response time, response length, and cosine similarity index (CSI) of their responses. Our findings show that Gemini consistently delivered more accurate responses to LLPS-related questions than ChatGPT. However, neither model delivered correct answers to all questions posed. Detailed analysis showed that Gemini required longer response times, averaging 272 words per response compared to ChatGPT’s 351. Additionally, the average CSI between the models was 0.62, highlighting moderate similarity. Despite both models showing potential to enhance scientific education in complex domains, our findings highlight a critical need for further refinement of these AI tools to improve their accuracy and reliability in specialized academic settings.

show abstract

An Extensive Comparative Analysis of Chatbot Technologies - ChatGPT, Google BARD and Microsoft Bing

Cited by 22 publications

References 0 publications

Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics

Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics

AI-Powered Western Blot Interpretation: A Novel Approach to Studying the Frameshift Mutant of Ubiquitin B (UBB+1) in Schizophrenia

AI for Biophysical Phenomena: A Comparative Study of ChatGPT and Gemini in Explaining Liquid–Liquid Phase Separation

Contact Info

Product

Resources

About