Are Preprints a Threat to the Credibility and Quality of Artificial Intelligence Literature in the ChatGPT Era? A Scoping Review and Qualitative Study

ChatGPT, as the pioneer of advanced generative AI tools, has triggered scholarly discussions about the potential use of such AI technologies in interdisciplinary fields. With a focus on the surge of AI-related preprints since the introduction of ChatGPT by OpenAI, the study investigated what the surge implies for AI literature, particularly in terms of credibility and quality. A scoping review was initially conducted to study the characteristics of the AI-related preprints in the Web of Science (WoS) database and also in five (5) preprint platforms (ArXiv, MedRxiv, SocArxiv, SSRN, and Research Square). The publication date range was set at January 01, 2023 to September 08, 2023. This was followed up by an interpretive phenomenological analysis (IPA) of the perceptions of experts in the AI field about the preprints. Employing a scoping review of AI-related preprints across six databases and a qualitative analysis of 15 AI experts' opinions, our study reveals concerns about the research accuracy, quality, and credibility of preprints, and advocates for a robust evaluation and high-quality assurance process to promote open science objectives during their dissemination. Specifically, 45,918 AI-related preprints were found in the six preprint databases or repositories across different fields. The nine themes from the IPA showed that preprints can be of value. However, experts advocated for the safe and responsible use of AI-related preprints, involving such tenets as maintaining ethical integrity and high-quality work on the part of authors and establishing sound AI-content guidelines from publishers and editors. Future studies are recommended to investigate the impact of preprints on decision-making processes in educational research and practice.