Implementing an online radiotherapy quality assurance programme with supporting continuous medical education - report from the EMBRACE-II evaluation of cervix cancer IMRT contouring

Background and purpose: EMBRACE-II is an international prospective study of IMRT and MRI-guided adaptive brachytherapy (IGABT) in locally advanced cervix cancer. An online radiotherapy quality assurance (RTQA) programme with minimal data transfer and supporting continuing medical education (CME) was implemented for IMRT contouring. Materials and methods: Participant contours for six volumes-of-interest (VOIs) on one benchmark case were scored (2 = excellent, 1 = fair, 0 = revision required) against a consensus reference contour. For contours receiving a 0 or 1 score, additional qualitative comments were provided. The Jaccard conformity index (JCI) was retrospectively calculated. User interaction with CME content (pre-accreditation questionnaire, contouring atlas, practice cases, quizzes, internal target volume (ITV-T) guide) was analysed. Results: 78 clinicians submitted contours for evaluation. 41% passed at the first attempt, 44% after one revision and 6% after two or more revisions. 9% did not re-submit after failing. The lowest mean scores were for the elective nodal CTV (CTV-E) (1.01/2) and ITV-T (1.06/2). 60 different errors across the six VOIs were identified; five potentially had high impact on loco-regional control. A JCI cut-off of 0.7 would have identified 87% contours that failed expert assessment, but also excluded 54% of passing contours. 39 clinicians responded to the pre-accreditation questionnaire - 36% anticipated difficulties with the ITV-T and 13% with the CTV-E. 35% clinicians contoured on the practice cases, 17% answered a quiz, 96% used the atlas and 38% the ITV-T guide. Conclusion: Expert evaluation with qualitative feedback improved contouring compliance. The JCI is not a reliable alternative to expert assessment. Moderate uptake of optional CME content limited evaluation. Crown Copyright (c) 2020 Published by Elsevier B.V. All rights reserved. Radiotherapy and Oncology 147 (2020) 22-29