Measuring the Effectiveness of AI Tools in Clinical Research and Writing: A Case Study in Healthcare

Abstract

This article investigates the capabilities and limitations of ChatGPT, a natural language processing (NLP) tool, and large language models (LLMs), developed from advanced artificial intelligence (AI). Designed to help computers understand and produce text understandable by humans, ChatGPT is particularly aimed at general scientific writing and healthcare research applications. Our methodology involved searching the Scopus database for ’type 2 diabetes’ and ’T2 diabetes’ articles from reputable journals. After eliminating duplicates, we used ChatGPT to formulate conclusions for each selected article by inputting their structured abstracts, excluding the original conclusions. Additionally, we tested ChatGPT’s response to simple misuse scenarios. Our findings show that ChatGPT can accurately grasp context and concisely summarize primary research findings. Additionally, it helps individuals who are not as experienced in mathematical analysis by providing coding guidelines for mathematical analyses in a variety of computer languages and by demystifying difficult model results. In conclusion, even if ChatGPT and other AI technologies are revolutionizing scientific publishing and healthcare, their use should be strictly controlled by authoritative laws.