Publishers’ and journals’ instructions to authors on use of generative artificial intelligence in academic and scientific publishing: bibliometric analysis

Abstract

Objectives To determine the extent and content of academic publishers’ and scientific journals’ guidance for authors on the use of generative artificial intelligence (GAI). Design Cross sectional, bibliometric study. Setting Websites of academic publishers and scientific journals, screened on 19-20 May 2023, with the search updated on 8-9 October 2023. Participants Top 100 largest academic publishers and top 100 highly ranked scientific journals, regardless of subject, language, or country of origin. Publishers were identified by the total number of journals in their portfolio, and journals were identified through the Scimago journal rank using the Hirsch index (H index) as an indicator of journal productivity and impact. Main outcome measures The primary outcomes were the content of GAI guidelines listed on the websites of the top 100 academic publishers and scientific journals, and the consistency of guidance between the publishers and their affiliated journals. Results Among the top 100 largest publishers, 24% provided guidance on the use of GAI, of which 15 (63%) were among the top 25 publishers. Among the top 100 highly ranked journals, 87% provided guidance on GAI. Of the publishers and journals with guidelines, the inclusion of GAI as an author was prohibited in 96% and 98%, respectively. Only one journal (1%) explicitly prohibited the use of GAI in the generation of a manuscript, and two (8%) publishers and 19 (22%) journals indicated that their guidelines exclusively applied to the writing process. When disclosing the use of GAI, 75% of publishers and 43% of journals included specific disclosure criteria. Where to disclose the use of GAI varied, including in the methods or acknowledgments, in the cover letter, or in a new section. Variability was also found in how to access GAI guidelines shared between journals and publishers. GAI guidelines in 12 journals directly conflicted with those developed by the publishers. The guidelines developed by top medical journals were broadly similar to those of academic journals. Conclusions Guidelines by some top publishers and journals on the use of GAI by authors are lacking. Among those that provided guidelines, the allowable uses of GAI and how it should be disclosed varied substantially, with this heterogeneity persisting in some instances among affiliated publishers and journals. Lack of standardization places a burden on authors and could limit the effectiveness of the regulations. As GAI continues to grow in popularity, standardized guidelines to protect the integrity of scientific output are needed.

A Systematic Examination of Generative Artificial Intelligence (GAI) Usage Guidelines for Scholarly Publishing in Medical Journals

“…[34][35][36] Interestingly, the figure we observed is considerably lower than that reported by another study (86%) focusing on only top journals. 30 Moreover, the provision of author guidelines among top medical journals (64%) is also lower in our study compared to that study. The main reason for the discrepancy in the provision of author guidelines between our study and the other is the different journal selection methods.…”

“…Together with two other recent studies showing inconsistencies in what to disclose of GAI usage, a need is highlighted surrounding standardization of GAI usage disclosure and documentation. 30,31,46 More importantly, journals held different and even opposite stances for content-generating applications, including manuscript writing, data analysis and interpretation, and image generating. This is due to concern centered on plagiarism because it is challenging to trace and verify the originality of AI-generated content.…”

“…For example, one study examined GAI usage guidelines for authors based on the 100 largest publishers and top 100 highly ranked journals of different disciplines, and it found less than a quarter of these publishers and around 85% of the top journals provided author guidelines. 30 The other study examined author disclosure requirements for GAI usage in 125 nursing journals and found less than 40% of them had explicit instructions. 31 They offered valuable insights into the evolving ethical standards and practices of this rapidly changing field.…”

Policy framework for the utilization of generative AI

“…In addition to the guidance of the above-mentioned agencies, recently, one study published by BMJ examined whether the top 100 academic publishers and the 100 most cited journals have established guidelines for the utilization of generative AI tools [ 4 ]. Among the top 100 publishers, 24 (24%) have issued directives regarding the use of GAI.…”

Evaluating ChatGPT-4.0’s data analytic proficiency in epidemiological studies: A comparative analysis with SAS, SPSS, and R

“…As one of the most advanced large language models (LLMs) available, ChatGPT-4 is noted for its user-friendly interface for human-computer interaction and its significant extension capabilities, allowing for the easy customisation of a ‘GPT’ to meet specific user needs. Consequently, it has been widely applied and researched across various domains [ 3 – 5 ], including academic research [ 6 – 8 ], health care analytics [ 9 – 11 ], and education [ 12 , 13 ].…”