An Advanced Machine Learning Model for a Web-Based Artificial Intelligence-Based Clinical Decision Support System Application: Model Development and Validation Study

Background: Breast cancer is a leading global health concern, necessitating advancements in recurrence prediction andmanagement. The development of an artificial intelligence (AI)-based clinical decision support system (AI-CDSS) using ChatGPTaddresses this need with the aim of enhancing both prediction accuracy and user accessibility. Objective: This study aims to develop and validate an advanced machine learning model for a web-based AI-CDSS application,leveraging the question-and-answer guidance capabilities of ChatGPT to enhance data preprocessing and model development,thereby improving the prediction of breast cancer recurrence. Methods: This study focused on developing an advanced machine learning model by leveraging data from the Tri-ServiceGeneral Hospital breast cancer registry of 3577 patients (2004-2016). As a tertiary medical center, it accepts referrals from fourbranches-3 branches in the northern region and 1 branch on an offshore island in our country-that manage chronic diseasesbut refer complex surgical cases, including breast cancer, to the main center, enriching our study population's diversity. Modeltraining used patient data from 2004 to 2012, with subsequent validation using data from 2013 to 2016, ensuring comprehensiveassessment and robustness of our predictive models. ChatGPT is integral to preprocessing and model development, aiding inhormone receptor categorization, age binning, and one-hot encoding. Techniques such as the synthetic minority oversamplingtechnique address the imbalance of data sets. Various algorithms, including light gradient-boosting machine, gradient boosting,and extreme gradient boosting, were used, and their performance was evaluated using metrics such as the area under the curve,accuracy, sensitivity, and F-1-score. Results: The light gradient-boosting machine model demonstrated superior performance, with an area under the curve of 0.80,followed closely by the gradient boosting and extreme gradient boosting models. The web interface of the AI-CDSS tool waseffectively tested in clinical decision-making scenarios, proving its use in personalized treatment planning and patient involvement. Conclusions: The AI-CDSS tool, enhanced by ChatGPT, marks a significant advancement in breast cancer recurrence prediction,offering a more individualized and accessible approach for clinicians and patients. Although promising, further validation indiverse clinical settings is recommended to confirm its efficacy and expand its use