Skin cancer classification using vision transformers and explainable artificial intelligence

Background: Skin cancer diagnosis is a critical aspect of dermatological healthcare, and requires accurate and efficient classification methods. Recently, vision transformers (ViTs) and convolutional neural networks (CNNs) have emerged as promising architectures. However, the interpretability of these models remains a concern, hindering their widespread adoption in the clinical setting. Therefore, the aim of this research is to propose an explainable skin cancer classification using deep learning and explainable artificial intelligence methods. Methods: This study presents skin cancer classification utilizing two pretrained ViTs, three Swin transformers, five pretrained CNNs, and three visual-based explainable artificial intelligence (XAI) models. The ViT-base, ViT-large, Swin-tiny, Swin-base, and Swin-small transformer models and VGG19, ResNet18, ResNet50, MobileNetV2, and DenseNet201 pretrained CNN models are used for classification. For explanation, gradient-weighted class activation mapping (Grad-CAM), Grad-CAM++, and score-weighted class activation mapping (Score-CAM) XAI models were adopted. The study used freely available datasets to train and test the proposed models and adopted synthetic minority oversampling technique (SMOTE) to address class imbalance issues. Results: The performances of the ViT and CNN models were evaluated using five performance metrics: accuracy, precision, F1 score, sensitivity, and specificity. The ViT models demonstrated competitive performance with ViT-large and ViT-base, achieving an accuracy of 88.6%. Swin-base exhibited a balanced sensitivity and specificity. Mainly, ResNet50 outperformed the tested models with an accuracy of 88.8%, precision of 86.9%, sensitivity of 88.6%, F1 score of 87.8%, and specificity of 88.9%. The integration of XAI techniques into the ResNet50 model showed that the model learns from relevant regions of the image to classify a given image as benign or malignant. Conclusions: This study presents ViT and CNN models for skin cancer classification, and the XAI techniques applied to the model contributes to enhancing transparency of the decision-making process of deep learning models. These findings will aid in accurate and trustworthy skin cancer classification and will be vital for clinical adoption in enhancing clinical decision-making in dermatological healthcare.