Late fusion of deep and shallow features to improve discrimination of actinic keratosis from normal skin using clinical photography

Background Actinic keratosis (AK) is a common premalignant skin lesion that can potentially progress to squamous cell carcinoma. Appropriate long-term management of AK requires close patient monitoring in addition to therapeutic interventions. Computer-aided diagnostic systems based on clinical photography might evolve in the future into valuable adjuncts to AK patient management. The present study proposes a late fusion approach of color-texture features (shallow features) and deep features extracted from pre-trained convolutional neural networks (CNN) to boost AK detection accuracy on clinical photographs. Materials and Methods System uses a sliding rectangular window of 50 x 50 pixels and a classifier that assigns the window region to either the AK or the healthy skin class. 6010 and 13 915 cropped regions of interest (ROI) of 50 x 50 pixels of AK and healthy skin, respectively, from 22 patients were used for system implementation. Different support vector machine (SVM) classifiers employing shallow or deep features and their late fusion using the max rule at decision level were compared with the McNemar test and Yule's Q-statistic. Results Support vector machine classifiers based on deep and shallow features exhibited overall competitive performances with complementary improvements in detection accuracy. Late fusion yielded significant improvement (6%) in both sensitivity (87%) and specificity (86%) compared to single classifier performance. Conclusion The parallel improvement of sensitivity and specificity is encouraging, demonstrating the potential use of our system in evaluating AK burden. The latter might be of value in future clinical studies for the comparison of field-directed treatment interventions.