Development and Evaluation of a Surveillance System for Follow-Up After Colorectal Polypectomy

IMPORTANCE The adherence of physicians and patients to published colorectal postpolypectomy surveillance guidelines varies greatly, and patient follow-up is critical but time consuming. OBJECTIVES To evaluate the accuracy of an automatic surveillance (AS) system in identifying patients after polypectomy, assigning surveillance intervals for different risks of patients, and proactively following up with patients on time. DESIGN, SETTING, AND PARTICIPANTS In this diagnostic/prognostic study, endoscopic and pathological reports of 47 544 patients undergoing colonoscopy at 3 hospitals between January 1, 2017, and June 30, 2022, were collected to develop an AS system based on natural language processing. The performance of the AS system was fully evaluated in internal and external tests according to 5 guidelines worldwide and compared with that of physicians. A multireader, multicase (MRMC) trial was conducted to evaluate use of the AS system and physician guideline adherence, and prospective data were collected to evaluate the success rate in contacting patients and the association with reduced human workload. Data analysis was conducted from July to September 2022. EXPOSURES Assistance of the AS system. MAIN OUTCOMES AND MEASURES The accuracy of the system in identifying patients after polypectomy, stratifying patient risk levels, and assigning surveillance intervals in internal (Renmin Hospital ofWuhan University), external 1 (Wenzhou Central Hospital), and external 2 (The First People's Hospital of Yichang) test sets; the accuracy of physicians and their time burden with and without system assistance; and the rate of successfully informed patients of the system were evaluated. RESULTS Test sets for 16 106 patients undergoing colonoscopy (mean [SD] age, 51.90 [13.40] years; 7690 females [47.75%]) were evaluated. In internal, external 1, and external 2 test sets, the system had an overall accuracy of 99.91%(95% CI, 99.83%-99.95%), 99.54%(95% CI, 99.30%-99.70%), and 99.77%(95% CI, 99.41%-99.91%), respectively, for identifying types of patients and achieved an overall accuracy of at least 99.30% (95% CI, 98.67%-99.63%) in the internal test set, 98.89%(95% CI, 98.33%-99.27%) in external test set 1, and 98.56%(95% CI, 95.86%-99.51%) in external test set 2 for stratifying patient risk levels and assigning surveillance intervals according to 5 guidelines. The system was associated with increased mean (SD) accuracy among physicians vs no AS system in 105 patients (98.67%[1.28%] vs 78.10% [18.01%]; P =.04) in the MRMC trial. In a prospective trial, the AS system successfully informed 82 of 88 patients (93.18%) and was associated with reduced burden of follow-up time vs no AS system (0 vs 2.86 h). CONCLUSIONS AND RELEVANCE This study found that an AS system was associated with improved adherence to guidelines among physicians and reduced workload among physicians and nurses.