Analysis of ChatGPT Tool to Assess the Potential of its Utility for Academic Writing in Biomedical Domain

Abstract

Background: ChatGPT (Generative Pre-trained Transformer) is an artificial intelligence chatbot which is designed to generate detailed text response and/or articulate answers to any query. Widespread concerns are expressed that such tools can be misused for academic writing. Hence this study was designed to test the utility and merit of ChatGPT for academic writing in biomedical sciences. Materials and Methods: five randomly selected quey topics were input into ChatGPT for a response. The ChatGPT response time, quality of the content and reliability were assessed. The ChatGPT response was transferred into a word file and the file was checked for originality using the Urkund software. Results: The response rate by ChatGPT was observed to be very quick i.e., less than 2 min to give a 300-500 words text output. Although the content of the response were systematic, precise and original, it lacked quality and depth of academic writing. Several shortcomings such as word count, referencing errors and lack of academic merit were observed. Conclusion: Despite its significant limitations, ChatGPT has enormous potential as training and upskilling resource for academic writing, which rather than replacing biological intelligence will help in refining it if used appropriately under academic mentoring.

ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns

“…Interestingly, a recent preprint by Benoit showed the promising potential of ChatGPT in rapidly crafting consistent realistic clinical vignettes of variable complexities that can be a valuable educational source with lower costs [ 67 ]. Thus, ChatGPT can be useful in health care education including enhanced communication skills given proper academic mentoring [ 42 , 57 , 67 ]. However, the copyright issues should be taken into account regarding the ChatGPT-generated clinical vignettes, aside from the issue of inaccurate references [ 67 ].…”

The Utility of ChatGPT as an Example of Large Language Models in Healthcare Education, Research and Practice: Systematic Review on the Future Perspectives and Potential Limitations

“…Interestingly, a recent preprint by Benoit showed the promising potential of ChatGPT in rapidly crafting consistent realistic clinical vignettes of variable complexities which can be a valuable educational source with lower costs [45]. Thus, ChatGPT can be useful in healthcare education including enhanced communication skills given proper academic mentoring [22,45,67]. However, the copyright issue in ChatGPT-generated clinical vignettes besides the issue of inaccurate references should be taken into account [45].…”

Human versus artificial intelligence‐generated arthroplasty literature: A single‐blinded analysis of perceived communication, quality, and authorship source

“…Though promulgated with varying degrees of hesitancy, speculative use-cases have spanned hospital discharge summaries, 16 medical information and patient medical record accessibility, 17,18 trainee education and evaluation, 19,20 and academic writing. 7,8,21,22 While a preponderance of commentaries/editorials on the topic have emerged since 2022, scientifically rigorous appraisal of LLMgenerated academic literature remains limited. 23 To our knowledge, only one such analysis has been conducted using ChatGPT, wherein Gao et al presented AI-generated and human-written abstracts to a group of blinded reviewers, and found that human reviewers were mostly successful in identifying on a binary scale abstract that had been written by AI, often noting them to be more vague.…”