Generative artificial intelligence (GAI) usage guidelines for scholarly publishing: a cross-sectional study of medical journals

Abstract

Background: Generative artificial intelligence (GAI) has developed rapidly and been increasingly used in scholarly publishing, so it is urgent to examine guidelines for its usage. This cross-sectional study aims to examine the coverage and type of recommendations of GAI usage guidelines among medical journals and how these factors relate to journal characteristics. Methods: From the SCImago Journal Rank (SJR) list for medicine in 2022, we generated two groups of journals: top SJR ranked journals (N = 200) and random sample of non-top SJR ranked journals (N = 140). For each group, we examined the coverage of author and reviewer guidelines across four categories: no guidelines, external guidelines only, own guidelines only, and own and external guidelines. We then calculated the number of recommendations by counting the number of usage recommendations for author and reviewer guidelines separately. Regression models examined the relationship of journal characteristics with the coverage and type of recommendations of GAI usage guidelines. Results: A higher proportion of top SJR ranked journals provided author guidelines compared to the random sample of non-top SJR ranked journals (95.0% vs. 86.7%, P < 0.01). The two groups of journals had the same median of 5 on a scale of 0 to 7 for author guidelines and a median of 1 on a scale of 0 to 2 for reviewer guidelines. However, both groups had lower percentages of journals providing recommendations for data analysis and interpretation, with the random sample of non-top SJR ranked journals having a significantly lower percentage (32.5% vs. 16.7%, P < 0.05). A higher SJR score was positively associated with providing GAI usage guidelines for both authors (all P < 0.01) and reviewers (all P < 0.01) among the random sample of non-top SJR ranked journals. Conclusions: Although most medical journals provided their own GAI usage guidelines or referenced external guidelines, some recommendations remained unspecified (e.g., whether AI can be used for data analysis and interpretation). Additionally, journals with lower SJR scores were less likely to provide guidelines, indicating a potential gap that warrants attention. Collaborative efforts are needed to develop specific recommendations that better guide authors and reviewers.